2024-04-25T00:28:28.971 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:28:28.971 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/workunit/dir-max-entries} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/orig.config.yaml 2024-04-25T00:28:29.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:28:29.105 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/workunit/dir-max-entries} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672276' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9cm981KG+zU/4NvE8B82NrdnmELXSd5LCeiKRt61nGN3j2j7n3e4/osGo0coIWjsx8Db8InqaJY4/MZpXbR18= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAlKPAhN/nGw+u89+i8y+9N/255AGygtVzmIqfMLXr7G5zDal68Y4TRlSwWuJlU68iHc3UAPN7dtKSvmiZaXrU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - workunit: clients: client.0: - fs/maxentries teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:28:29.105 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:28:29.106 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:28:29.106 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:28:29.107 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:28:29.131 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:28:29.131 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:28:29.131 INFO:teuthology.packaging:ref: None 2024-04-25T00:28:29.131 INFO:teuthology.packaging:tag: None 2024-04-25T00:28:29.131 INFO:teuthology.packaging:branch: squid 2024-04-25T00:28:29.132 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:28:29.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-25T00:28:29.432 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:28:29.434 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:28:29.441 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:28:29.441 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:28:29.448 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:28:29.465 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:28:29.471 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:28:29.496 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:23:01.222786', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAAs4PTlN+e3thnyOXPNC6K7FT2HdnYmke99r/VSzxg5'} 2024-04-25T00:28:29.518 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:23:01.220930', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKBSa1j7/4Zp2L/TKHjGMXJejBDomy8KPQkSTTaqUdOq'} 2024-04-25T00:28:29.518 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:28:29.525 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T00:28:29.525 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T00:28:29.525 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:28:29.600 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8a4cfb1ee0>, signals=[15]) 2024-04-25T00:28:29.600 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:28:29.608 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:28:29.608 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:28:29.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:28:29.697 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:28:29.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:28:29.774 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:28:29.781 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-04-25T00:28:29.786 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-04-25T00:28:29.786 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-04-25T00:28:29.832 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="Ubuntu" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="22.04" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_CODENAME=jammy 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:ID=ubuntu 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE=debian 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:28:29.833 INFO:teuthology.orchestra.run.smithi129.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:28:29.834 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-04-25T00:28:29.862 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-25T00:28:29.867 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-25T00:28:29.867 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Ubuntu" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="22.04" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_CODENAME=jammy 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:ID=ubuntu 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE=debian 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:28:29.914 INFO:teuthology.orchestra.run.smithi196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:28:29.915 INFO:teuthology.orchestra.run.smithi196.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:28:29.915 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-25T00:28:29.937 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:28:29.946 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:28:29.952 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:28:29.952 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:28:29.954 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:28:29.961 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:28:29.968 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:28:29.968 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:28:30.002 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:28:30.011 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:28:30.069 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:28:30.083 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:28:30.084 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T00:28:30.084 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:28:30.085 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:28:30.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:28:30.089 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-25T00:28:30.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:28:30.090 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-04-25T00:28:30.134 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-56-generic 2024-04-25T00:28:30.135 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.15.0-56-generic 2024-04-25T00:28:30.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2024-04-25T00:28:30.136 INFO:teuthology.orchestra.run.smithi129.stdout:5.15.0-56-generic 2024-04-25T00:28:30.136 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.15.0-56-generic 2024-04-25T00:28:30.137 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-get clean 2024-04-25T00:28:30.264 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-25T00:28:30.272 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-get update 2024-04-25T00:28:30.406 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:28:30.409 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:28:30.412 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:28:30.412 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:28:30.423 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:28:30.500 INFO:teuthology.orchestra.run.smithi129.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:28:30.502 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:28:30.512 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:28:30.633 INFO:teuthology.orchestra.run.smithi129.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:28:30.649 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:28:30.723 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:28:30.743 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:28:30.745 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:28:30.754 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:28:30.757 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:28:30.758 INFO:teuthology.orchestra.run.smithi129.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:28:30.758 INFO:teuthology.orchestra.run.smithi129.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:28:30.792 INFO:teuthology.orchestra.run.smithi129.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:28:30.796 INFO:teuthology.orchestra.run.smithi129.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:28:30.797 INFO:teuthology.orchestra.run.smithi129.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:28:30.806 INFO:teuthology.orchestra.run.smithi129.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:28:30.814 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:28:30.821 INFO:teuthology.orchestra.run.smithi129.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:28:30.824 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:28:30.824 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:28:30.824 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:28:30.825 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:28:30.842 INFO:teuthology.orchestra.run.smithi129.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:28:30.843 INFO:teuthology.orchestra.run.smithi129.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:28:30.849 INFO:teuthology.orchestra.run.smithi129.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:28:30.856 INFO:teuthology.orchestra.run.smithi129.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:28:30.860 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:28:30.861 INFO:teuthology.orchestra.run.smithi129.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:28:30.861 INFO:teuthology.orchestra.run.smithi129.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:28:30.863 INFO:teuthology.orchestra.run.smithi129.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:28:30.864 INFO:teuthology.orchestra.run.smithi129.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:28:30.865 INFO:teuthology.orchestra.run.smithi129.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:28:30.866 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:28:30.866 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:28:30.873 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:28:30.890 INFO:teuthology.orchestra.run.smithi129.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:28:30.892 INFO:teuthology.orchestra.run.smithi129.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:28:30.892 INFO:teuthology.orchestra.run.smithi129.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:28:30.892 INFO:teuthology.orchestra.run.smithi129.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:28:30.915 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:28:30.915 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:28:30.917 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:28:30.918 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:28:30.918 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:28:30.918 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:28:30.919 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:28:30.921 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:28:30.921 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:28:30.951 INFO:teuthology.orchestra.run.smithi129.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:28:30.952 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:28:30.953 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:28:30.958 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:28:30.958 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:28:30.967 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:28:30.978 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:28:30.981 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:28:30.981 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:28:30.982 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:28:30.989 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:28:30.990 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:28:30.990 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:28:30.991 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:28:31.003 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:28:31.003 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:28:31.004 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:28:31.004 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:28:31.004 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:28:31.010 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:28:31.363 INFO:teuthology.orchestra.run.smithi129.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:28:31.380 INFO:teuthology.orchestra.run.smithi129.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:28:31.390 INFO:teuthology.orchestra.run.smithi129.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:28:31.390 INFO:teuthology.orchestra.run.smithi129.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:28:31.484 INFO:teuthology.orchestra.run.smithi129.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:28:31.485 INFO:teuthology.orchestra.run.smithi129.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:28:31.494 INFO:teuthology.orchestra.run.smithi129.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:28:31.495 INFO:teuthology.orchestra.run.smithi129.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:28:31.515 INFO:teuthology.orchestra.run.smithi129.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:28:31.695 INFO:teuthology.orchestra.run.smithi129.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:28:32.104 INFO:teuthology.orchestra.run.smithi129.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:28:32.110 INFO:teuthology.orchestra.run.smithi129.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:28:32.110 INFO:teuthology.orchestra.run.smithi129.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:28:32.124 INFO:teuthology.orchestra.run.smithi129.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:28:32.128 INFO:teuthology.orchestra.run.smithi129.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:28:36.325 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 13.3 MB in 2s (7,195 kB/s) 2024-04-25T00:28:36.678 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 13.3 MB in 2s (7,168 kB/s) 2024-04-25T00:28:37.682 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:28:37.700 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:28:37.763 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:28:37.943 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T00:28:37.944 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T00:28:38.075 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:28:38.094 DEBUG:teuthology.orchestra.run.smithi129:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:28:38.100 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:28:38.100 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:28:38.100 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:28:38.101 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:28:38.101 INFO:teuthology.orchestra.run.smithi196.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:28:38.101 INFO:teuthology.orchestra.run.smithi196.stdout: python2.7-minimal 2024-04-25T00:28:38.101 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:28:38.102 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-25T00:28:38.102 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:28:38.102 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:28:38.102 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:28:38.103 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-25T00:28:38.103 INFO:teuthology.orchestra.run.smithi196.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:28:38.103 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-04-25T00:28:38.104 INFO:teuthology.orchestra.run.smithi196.stdout: thermald 2024-04-25T00:28:38.148 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-25T00:28:38.148 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:28:38.148 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:28:38.149 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:28:38.150 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-04-25T00:28:38.150 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:28:38.153 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:28:38.226 INFO:teuthology.orchestra.run.smithi196.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:28:38.226 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 113 MB of archives. 2024-04-25T00:28:38.226 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:28:38.226 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:28:38.334 INFO:teuthology.orchestra.run.smithi129.stdout:Building dependency tree... 2024-04-25T00:28:38.335 INFO:teuthology.orchestra.run.smithi129.stdout:Reading state information... 2024-04-25T00:28:38.492 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:28:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:28:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:28:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:28:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:28:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: python2.7-minimal 2024-04-25T00:28:38.493 INFO:teuthology.orchestra.run.smithi129.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:28:38.494 INFO:teuthology.orchestra.run.smithi129.stdout:The following additional packages will be installed: 2024-04-25T00:28:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:28:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:28:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:28:38.495 INFO:teuthology.orchestra.run.smithi129.stdout:Suggested packages: 2024-04-25T00:28:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:28:38.495 INFO:teuthology.orchestra.run.smithi129.stdout:Recommended packages: 2024-04-25T00:28:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: thermald 2024-04-25T00:28:38.541 INFO:teuthology.orchestra.run.smithi129.stdout:The following NEW packages will be installed: 2024-04-25T00:28:38.542 INFO:teuthology.orchestra.run.smithi129.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:28:38.542 INFO:teuthology.orchestra.run.smithi129.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:28:38.542 INFO:teuthology.orchestra.run.smithi129.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:28:38.543 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages will be upgraded: 2024-04-25T00:28:38.544 INFO:teuthology.orchestra.run.smithi129.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:28:38.631 INFO:teuthology.orchestra.run.smithi129.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:28:38.631 INFO:teuthology.orchestra.run.smithi129.stdout:Need to get 113 MB of archives. 2024-04-25T00:28:38.631 INFO:teuthology.orchestra.run.smithi129.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:28:38.631 INFO:teuthology.orchestra.run.smithi129.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:28:39.453 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:28:39.827 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:28:39.869 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:28:40.167 INFO:teuthology.orchestra.run.smithi129.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:28:42.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:28:42.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:28:42.077 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:28:42.111 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:28:42.111 INFO:teuthology.orchestra.run.smithi129.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:28:42.112 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:28:42.309 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:28:42.373 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:28:42.377 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:28:42.426 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:28:42.763 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 113 MB in 4s (26.9 MB/s) 2024-04-25T00:28:42.818 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 113 MB in 4s (29.2 MB/s) 2024-04-25T00:28:42.946 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:28:42.993 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:28:44.852 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:28:44.856 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:28:44.951 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:28:44.953 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:28:44.958 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:28:45.014 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:28:47.230 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:28:47.248 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:28:47.331 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:28:47.665 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:28:47.687 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:28:47.773 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:28:47.789 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:28:47.809 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:28:47.847 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:28:48.260 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:28:48.280 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:28:48.318 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:28:54.386 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:28:54.490 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:28:54.505 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:28:54.592 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:28:54.823 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:28:54.883 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:28:54.944 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:28:55.012 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:28:55.247 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:28:55.258 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:28:55.297 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:28:55.316 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:28:55.338 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:28:55.382 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:29:00.748 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:29:00.785 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:29:00.923 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:29:00.960 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:29:00.989 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:01.056 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:03.784 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:29:03.910 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:29:04.108 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:29:04.234 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:29:04.251 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:29:04.373 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:04.499 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:29:04.567 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:29:04.627 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:04.688 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:04.823 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:29:04.951 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:05.954 INFO:teuthology.orchestra.run.smithi196.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:29:05.954 INFO:teuthology.orchestra.run.smithi196.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:29:05.954 INFO:teuthology.orchestra.run.smithi196.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:29:05.954 INFO:teuthology.orchestra.run.smithi196.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:06.112 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:06.355 INFO:teuthology.orchestra.run.smithi129.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:29:06.355 INFO:teuthology.orchestra.run.smithi129.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:29:06.355 INFO:teuthology.orchestra.run.smithi129.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:29:06.355 INFO:teuthology.orchestra.run.smithi129.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:06.662 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:07.515 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:08.122 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:08.940 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:29:09.068 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:29:09.195 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:09.259 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:29:09.259 INFO:teuthology.orchestra.run.smithi196.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:09.548 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:29:09.682 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:29:09.808 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:29:09.869 INFO:teuthology.orchestra.run.smithi129.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:29:09.870 INFO:teuthology.orchestra.run.smithi129.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:29:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:29:23.356 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:29:23.379 INFO:teuthology.orchestra.run.smithi196.stdout:Generating grub configuration file ... 2024-04-25T00:29:23.645 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:29:23.661 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:23.959 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:29:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:29:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:29:23.964 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:29:23.972 INFO:teuthology.orchestra.run.smithi129.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:29:23.995 INFO:teuthology.orchestra.run.smithi129.stdout:Generating grub configuration file ... 2024-04-25T00:29:24.036 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:29:24.041 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:29:24.213 INFO:teuthology.orchestra.run.smithi196.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:29:24.213 INFO:teuthology.orchestra.run.smithi196.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:29:24.213 INFO:teuthology.orchestra.run.smithi196.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:29:24.258 INFO:teuthology.orchestra.run.smithi129.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:29:24.258 INFO:teuthology.orchestra.run.smithi196.stdout:done 2024-04-25T00:29:24.273 INFO:teuthology.orchestra.run.smithi129.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:24.567 INFO:teuthology.orchestra.run.smithi129.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:29:24.573 INFO:teuthology.orchestra.run.smithi129.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:29:24.642 INFO:teuthology.orchestra.run.smithi129.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:29:24.647 INFO:teuthology.orchestra.run.smithi129.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:29:24.715 DEBUG:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 21 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2024-04-25T00:29:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.15.0.105.102 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:29:24.734 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2024-04-25T00:29:24.734 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:29:24.734 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:29:24.734 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:29:24.735 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T00:29:24.735 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:29:24.735 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2024-04-25T00:29:24.795 INFO:teuthology.orchestra.run.smithi129.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:29:24.795 INFO:teuthology.orchestra.run.smithi129.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:29:24.795 INFO:teuthology.orchestra.run.smithi129.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:29:24.842 INFO:teuthology.orchestra.run.smithi129.stdout:done 2024-04-25T00:29:24.916 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-25T00:29:25.036 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:29:25.036 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:29:25.045 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:29:25.065 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:29:25.292 DEBUG:teuthology.orchestra.run.smithi129:> dpkg -s linux-image-generic 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Package: linux-image-generic 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Status: install ok installed 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Priority: optional 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Section: kernel 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Installed-Size: 21 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: amd64 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Source: linux-meta 2024-04-25T00:29:25.313 INFO:teuthology.orchestra.run.smithi129.stdout:Version: 5.15.0.105.102 2024-04-25T00:29:25.314 INFO:teuthology.orchestra.run.smithi129.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:29:25.314 INFO:teuthology.orchestra.run.smithi129.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:29:25.314 INFO:teuthology.orchestra.run.smithi129.stdout:Recommends: thermald 2024-04-25T00:29:25.314 INFO:teuthology.orchestra.run.smithi129.stdout:Description: Generic Linux kernel image 2024-04-25T00:29:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:29:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: available. 2024-04-25T00:29:25.314 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:29:25.314 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:29:25.314 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:29:25.314 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T00:29:25.314 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:29:25.315 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-get clean 2024-04-25T00:29:25.474 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-get update 2024-04-25T00:29:25.691 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:29:25.814 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:29:25.885 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:29:25.966 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:29:26.397 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:29:26.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:29:26.475 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:29:26.679 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T00:29:26.680 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T00:29:26.849 INFO:teuthology.orchestra.run.smithi196.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:29:26.849 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:29:26.849 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:29:26.849 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:29:26.849 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:29:26.850 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:29:26.850 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:29:26.850 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:29:26.888 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:29:26.890 DEBUG:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 21 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:29:26.902 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.15.0.105.102 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:29:26.903 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2024-04-25T00:29:26.903 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-25T00:29:26.948 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.mq09ujI9JF 2024-04-25T00:29:26.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub/grub.cfg /tmp/tmp.mq09ujI9JF 2024-04-25T00:29:27.007 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.mq09ujI9JF 2024-04-25T00:29:27.083 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:29:27.101 DEBUG:teuthology.orchestra.run.smithi129:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:29:27.120 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.mq09ujI9JF is 10KB 2024-04-25T00:29:27.132 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.mq09ujI9JF 2024-04-25T00:29:27.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:29:27.152 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:29:27.192 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:29:27.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:29:27.192 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:29:27.254 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:29:27.255 DEBUG:teuthology.orchestra.run.smithi196:> sudo update-grub 2024-04-25T00:29:27.335 INFO:teuthology.orchestra.run.smithi129.stdout:Building dependency tree... 2024-04-25T00:29:27.335 INFO:teuthology.orchestra.run.smithi129.stdout:Reading state information... 2024-04-25T00:29:27.493 INFO:teuthology.orchestra.run.smithi129.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:29:27.493 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:29:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:29:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:29:27.493 INFO:teuthology.orchestra.run.smithi129.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:29:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:29:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:29:27.494 INFO:teuthology.orchestra.run.smithi129.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:29:27.532 INFO:teuthology.orchestra.run.smithi129.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:29:27.535 DEBUG:teuthology.orchestra.run.smithi129:> dpkg -s linux-image-generic 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Package: linux-image-generic 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Status: install ok installed 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Priority: optional 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Section: kernel 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Installed-Size: 21 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: amd64 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Source: linux-meta 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Version: 5.15.0.105.102 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:29:27.590 INFO:teuthology.orchestra.run.smithi129.stdout:Recommends: thermald 2024-04-25T00:29:27.591 INFO:teuthology.orchestra.run.smithi129.stdout:Description: Generic Linux kernel image 2024-04-25T00:29:27.591 INFO:teuthology.orchestra.run.smithi129.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:29:27.591 INFO:teuthology.orchestra.run.smithi129.stdout: available. 2024-04-25T00:29:27.592 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-04-25T00:29:27.639 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.3jyoZo88yz 2024-04-25T00:29:27.640 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub/grub.cfg /tmp/tmp.3jyoZo88yz 2024-04-25T00:29:27.696 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.3jyoZo88yz 2024-04-25T00:29:27.807 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.3jyoZo88yz is 10KB 2024-04-25T00:29:27.820 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.3jyoZo88yz 2024-04-25T00:29:27.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:29:27.878 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:29:27.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:29:27.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:29:27.943 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:29:27.944 DEBUG:teuthology.orchestra.run.smithi129:> sudo update-grub 2024-04-25T00:29:28.393 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:29:28.394 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:29:28.398 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-04-25T00:29:28.558 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:29:28.563 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:28.834 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:29:28.839 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:29:28.879 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:29:28.884 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:29:29.034 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:29:29.034 INFO:teuthology.orchestra.run.smithi196.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:29:29.034 INFO:teuthology.orchestra.run.smithi196.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:29:29.049 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-04-25T00:29:29.051 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-04-25T00:29:29.103 INFO:teuthology.orchestra.run.smithi129.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:29:29.105 INFO:teuthology.orchestra.run.smithi129.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:29:29.107 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-04-25T00:29:29.290 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:29:29.297 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:29:29.531 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:29:29.536 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:29:29.572 INFO:teuthology.orchestra.run.smithi129.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:29:29.577 INFO:teuthology.orchestra.run.smithi129.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:29:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:29:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:29:29.694 INFO:teuthology.orchestra.run.smithi129.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:29:29.703 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-04-25T00:29:29.711 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-04-25T00:29:59.082 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T00:29:59.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:29:59.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:29:59.714 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T00:29:59.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:29:59.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:30:17.635 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-25T00:30:18.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-04-25T00:30:26.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:30:26.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:30:27.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:30:27.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:30:29.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-25T00:30:30.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-04-25T00:30:41.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:30:41.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:30:42.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:30:42.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:31:00.127 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-25T00:31:00.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-04-25T00:31:15.139 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:31:15.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:31:15.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-25T00:31:15.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:31:15.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:31:15.870 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-04-25T00:31:16.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:31:16.472 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T00:31:16.472 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-04-25T00:31:16.518 INFO:teuthology.orchestra.run.smithi129.stdout:5.15.0-105-generic 2024-04-25T00:31:16.518 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:31:16.519 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:31:16.519 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:31:17.519 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T00:31:17.520 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:31:17.589 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-04-25T00:31:17.612 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:33.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:31:33.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:31:33.489 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-25T00:31:34.140 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:31:34.140 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T00:31:34.140 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-25T00:31:34.185 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-105-generic 2024-04-25T00:31:34.185 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:31:34.185 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:31:34.186 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:31:35.186 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T00:31:35.187 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:31:35.539 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-04-25T00:31:35.563 DEBUG:teuthology.parallel:result is None 2024-04-25T00:31:35.563 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:31:35.596 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:31:35.596 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:31:35.598 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:31:35.604 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:31:35.612 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:31:35.623 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:31:35.624 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:31:35.647 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:31:35.678 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:31:35.692 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:31:35.692 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:31:35.711 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:31:35.731 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:31:35.738 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:31:35.743 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:31:35.751 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:31:35.752 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:31:35.761 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:31:35.761 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:31:35.788 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:31:35.811 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:31:35.823 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:31:35.823 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:31:35.856 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:31:35.861 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:31:35.907 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:31:35.955 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:31:35.955 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:31:36.017 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:31:36.022 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:31:36.070 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:31:36.070 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:31:36.138 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-04-25T00:31:36.141 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-25T00:31:36.261 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:31:36.270 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:31:36.270 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:31:36.281 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:31:36.296 DEBUG:teuthology.task.selinux:Excluding smithi129: OS 'ubuntu' does not support SELinux 2024-04-25T00:31:36.296 DEBUG:teuthology.task.selinux:Excluding smithi196: OS 'ubuntu' does not support SELinux 2024-04-25T00:31:36.297 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:31:36.297 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T00:31:36.297 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:31:36.297 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:31:36.317 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:31:36.333 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:31:36.334 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:38:39.845 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-04-25T00:38:39.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:38:39.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:38:39.930 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-04-25T00:38:40.006 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:38:40.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:38:40.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:38:40.085 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-25T00:38:40.162 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T00:38:40.163 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:38:40.175 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:38:40.175 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:38:40.176 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:38:40.178 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:38:40.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:38:40.202 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:38:40.202 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Command line: ntpd -gq 2024-04-25T00:38:40.202 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: ---------------------------------------------------- 2024-04-25T00:38:40.203 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:38:40.203 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:38:40.203 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: corporation. Support and training for ntp-4 are 2024-04-25T00:38:40.203 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: available at https://www.nwtime.org/support 2024-04-25T00:38:40.203 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: ---------------------------------------------------- 2024-04-25T00:38:40.203 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: proto: precision = 0.044 usec (-24) 2024-04-25T00:38:40.204 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: basedate set to 2022-02-04 2024-04-25T00:38:40.204 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:38:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:38:40.204 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:38:40.204 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:38:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:38:40.205 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:38:40.206 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:38:40.206 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:38:40.206 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listen normally on 3 enp3s0f1 172.21.15.129:123 2024-04-25T00:38:40.206 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listen normally on 4 lo [::1]:123 2024-04-25T00:38:40.206 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7261%5]:123 2024-04-25T00:38:40.206 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:40 ntpd[17200]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:38:40.231 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:38:40.231 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Command line: ntpd -gq 2024-04-25T00:38:40.232 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: ---------------------------------------------------- 2024-04-25T00:38:40.232 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:38:40.232 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:38:40.232 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: corporation. Support and training for ntp-4 are 2024-04-25T00:38:40.232 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: available at https://www.nwtime.org/support 2024-04-25T00:38:40.232 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: ---------------------------------------------------- 2024-04-25T00:38:40.233 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: proto: precision = 0.060 usec (-24) 2024-04-25T00:38:40.233 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: basedate set to 2022-02-04 2024-04-25T00:38:40.233 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:38:40.233 INFO:teuthology.orchestra.run.smithi196.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:38:40.233 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:38:40.233 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:38:40.234 INFO:teuthology.orchestra.run.smithi196.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listen normally on 3 enp3s0f1 172.21.15.196:123 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listen normally on 4 lo [::1]:123 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f47%5]:123 2024-04-25T00:38:40.235 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:40 ntpd[17195]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:38:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:41 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:41.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:41 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:42.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:42 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:42.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:42 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:42.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:42 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:42.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:42 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:42.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:42 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:42.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:42 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:42.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:42 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:42.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:42 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:42.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:42 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:42.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:42 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:42.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:42 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:42.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:42 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:43.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:43 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:43.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:43 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:44.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:44 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:44.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:44.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:44.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:44.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:44.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:44 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:45.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:45 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:45.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:45 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:45.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:45 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:45.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:45 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:45.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:45 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:45.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:45 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:45.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:45 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:45.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:45 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:45.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:45 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:45.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:45 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:45.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:45 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:45.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:45 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:46.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:46.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:46.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:46 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:46.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:46 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:47.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:47 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:47.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:47 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:47.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:47 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:47.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:47 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:47.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:47 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:47.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:47 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:47.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:47 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:47.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:47 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:47.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:47 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:47.232 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:47 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:47.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:47 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:47.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:47 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:48.203 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stdout:25 Apr 00:38:48 ntpd[17200]: ntpd: time slew -0.001290 s 2024-04-25T00:38:48.204 INFO:teuthology.orchestra.run.smithi129.stdout:ntpd: time slew -0.001290s 2024-04-25T00:38:48.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:38:48.205 INFO:teuthology.orchestra.run.smithi129.stderr:25 Apr 00:38:48 ntpd[17200]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:38:48.233 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:38:48.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:38:48.234 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 00:38:48 ntpd[17195]: ntpd: time slew -0.001217 s 2024-04-25T00:38:48.234 INFO:teuthology.orchestra.run.smithi196.stdout:ntpd: time slew -0.001217s 2024-04-25T00:38:48.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:38:48.234 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 00:38:48 ntpd[17195]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:38:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:38:48.256 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================== 2024-04-25T00:38:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.282 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:38:48.282 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-04-25T00:38:48.282 INFO:teuthology.orchestra.run.smithi196.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.282 INFO:teuthology.orchestra.run.smithi196.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.282 INFO:teuthology.orchestra.run.smithi196.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.282 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:38:48.283 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:38:48.293 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:38:48.293 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:38:48.293 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:38:48.293 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:38:48.298 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:38:48.299 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:38:48.299 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-key list | grep Ceph 2024-04-25T00:38:48.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-key list | grep Ceph 2024-04-25T00:38:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:38:48.382 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:38:48.386 INFO:teuthology.orchestra.run.smithi129.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:38:48.386 INFO:teuthology.orchestra.run.smithi129.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:38:48.387 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:38:48.387 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:38:48.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:38:48.410 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:38:48.410 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:38:48.411 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:38:48.411 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:38:48.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:38:48.553 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:38:48.553 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:38:48.601 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:38:48.601 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:38:48.666 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:38:48.666 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:38:48.680 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-25T00:38:48.707 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:38:48.707 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:38:48.722 DEBUG:teuthology.orchestra.run.smithi129:> sudo apt-get update 2024-04-25T00:38:48.851 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:38:48.852 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:38:48.863 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:38:48.883 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:38:48.899 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:38:48.920 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:38:48.940 INFO:teuthology.orchestra.run.smithi129.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:38:48.941 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:38:48.962 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:38:48.963 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:38:48.985 INFO:teuthology.orchestra.run.smithi129.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:38:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:38:49.006 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:38:49.025 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:38:49.025 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:38:49.046 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:38:49.090 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:38:49.209 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 28.9 kB in 0s (82.6 kB/s) 2024-04-25T00:38:49.439 INFO:teuthology.orchestra.run.smithi129.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:38:49.594 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 258 kB in 1s (376 kB/s) 2024-04-25T00:38:50.194 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:38:50.211 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:38:50.264 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:38:50.442 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T00:38:50.442 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T00:38:50.591 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:38:50.607 DEBUG:teuthology.orchestra.run.smithi129:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:38:50.614 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:38:50.614 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:38:50.614 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:38:50.614 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:38:50.614 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:38:50.615 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:38:50.615 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:38:50.616 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-25T00:38:50.616 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:38:50.616 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:38:50.616 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:38:50.616 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:38:50.616 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:38:50.617 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:38:50.618 INFO:teuthology.orchestra.run.smithi196.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:38:50.619 INFO:teuthology.orchestra.run.smithi196.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:38:50.619 INFO:teuthology.orchestra.run.smithi196.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:38:50.619 INFO:teuthology.orchestra.run.smithi196.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:38:50.619 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-04-25T00:38:50.619 INFO:teuthology.orchestra.run.smithi196.stdout: btrfs-tools 2024-04-25T00:38:50.659 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:38:50.724 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:38:50.725 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:38:50.726 INFO:teuthology.orchestra.run.smithi196.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:38:50.726 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:38:50.726 INFO:teuthology.orchestra.run.smithi196.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:38:50.726 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:38:50.726 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:38:50.726 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:38:50.727 INFO:teuthology.orchestra.run.smithi196.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:38:50.728 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:38:50.728 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:38:50.728 INFO:teuthology.orchestra.run.smithi196.stdout: socat unzip xmlstarlet zip 2024-04-25T00:38:50.791 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T00:38:50.791 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 180 MB of archives. 2024-04-25T00:38:50.791 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:38:50.791 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:38:50.812 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:38:50.824 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:38:50.836 INFO:teuthology.orchestra.run.smithi129.stdout:Building dependency tree... 2024-04-25T00:38:50.837 INFO:teuthology.orchestra.run.smithi129.stdout:Reading state information... 2024-04-25T00:38:50.873 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:38:50.898 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:38:50.901 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:38:50.915 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:38:50.943 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:38:50.966 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:38:50.970 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:38:50.982 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:38:50.986 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:38:50.986 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:38:50.994 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:38:50.996 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:38:50.999 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:38:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:38:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:38:51.008 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:38:51.008 INFO:teuthology.orchestra.run.smithi129.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:38:51.008 INFO:teuthology.orchestra.run.smithi129.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:38:51.008 INFO:teuthology.orchestra.run.smithi129.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:38:51.008 INFO:teuthology.orchestra.run.smithi129.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:38:51.010 INFO:teuthology.orchestra.run.smithi129.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout:The following additional packages will be installed: 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:38:51.011 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:38:51.012 INFO:teuthology.orchestra.run.smithi129.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi129.stdout:Suggested packages: 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi129.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi129.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi129.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi129.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:38:51.013 INFO:teuthology.orchestra.run.smithi129.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:38:51.014 INFO:teuthology.orchestra.run.smithi129.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:38:51.014 INFO:teuthology.orchestra.run.smithi129.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:38:51.014 INFO:teuthology.orchestra.run.smithi129.stdout:Recommended packages: 2024-04-25T00:38:51.014 INFO:teuthology.orchestra.run.smithi129.stdout: btrfs-tools 2024-04-25T00:38:51.014 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:38:51.015 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:38:51.015 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:38:51.032 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:38:51.033 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:38:51.033 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:38:51.033 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:38:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:38:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:38:51.052 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:38:51.052 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:38:51.055 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:38:51.055 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:38:51.072 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:38:51.073 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:38:51.074 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:38:51.074 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:38:51.074 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:38:51.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:38:51.093 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:38:51.094 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:38:51.099 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:38:51.099 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:38:51.113 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:38:51.114 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:38:51.115 INFO:teuthology.orchestra.run.smithi196.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:38:51.117 INFO:teuthology.orchestra.run.smithi196.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:38:51.118 INFO:teuthology.orchestra.run.smithi196.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:38:51.119 INFO:teuthology.orchestra.run.smithi196.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:38:51.120 INFO:teuthology.orchestra.run.smithi129.stdout:The following NEW packages will be installed: 2024-04-25T00:38:51.120 INFO:teuthology.orchestra.run.smithi129.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:38:51.120 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:38:51.120 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:38:51.122 INFO:teuthology.orchestra.run.smithi129.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:38:51.123 INFO:teuthology.orchestra.run.smithi129.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:38:51.124 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:38:51.124 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:38:51.124 INFO:teuthology.orchestra.run.smithi129.stdout: socat unzip xmlstarlet zip 2024-04-25T00:38:51.135 INFO:teuthology.orchestra.run.smithi196.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:38:51.137 INFO:teuthology.orchestra.run.smithi196.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:38:51.137 INFO:teuthology.orchestra.run.smithi196.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:38:51.138 INFO:teuthology.orchestra.run.smithi196.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:38:51.155 INFO:teuthology.orchestra.run.smithi196.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:38:51.206 INFO:teuthology.orchestra.run.smithi196.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:38:51.207 INFO:teuthology.orchestra.run.smithi196.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:38:51.208 INFO:teuthology.orchestra.run.smithi196.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:38:51.208 INFO:teuthology.orchestra.run.smithi196.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:38:51.209 INFO:teuthology.orchestra.run.smithi196.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:38:51.214 INFO:teuthology.orchestra.run.smithi129.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T00:38:51.214 INFO:teuthology.orchestra.run.smithi129.stdout:Need to get 180 MB of archives. 2024-04-25T00:38:51.214 INFO:teuthology.orchestra.run.smithi129.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:38:51.214 INFO:teuthology.orchestra.run.smithi129.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:38:51.230 INFO:teuthology.orchestra.run.smithi196.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:38:51.232 INFO:teuthology.orchestra.run.smithi196.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:38:51.232 INFO:teuthology.orchestra.run.smithi196.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:38:51.245 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:38:51.253 INFO:teuthology.orchestra.run.smithi196.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:38:51.253 INFO:teuthology.orchestra.run.smithi196.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:38:51.255 INFO:teuthology.orchestra.run.smithi129.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:38:51.262 INFO:teuthology.orchestra.run.smithi196.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:38:51.262 INFO:teuthology.orchestra.run.smithi196.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:38:51.263 INFO:teuthology.orchestra.run.smithi196.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:38:51.263 INFO:teuthology.orchestra.run.smithi196.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:38:51.264 INFO:teuthology.orchestra.run.smithi196.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:38:51.264 INFO:teuthology.orchestra.run.smithi196.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:38:51.268 INFO:teuthology.orchestra.run.smithi196.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:38:51.282 INFO:teuthology.orchestra.run.smithi196.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:38:51.284 INFO:teuthology.orchestra.run.smithi196.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:38:51.284 INFO:teuthology.orchestra.run.smithi196.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:38:51.298 INFO:teuthology.orchestra.run.smithi196.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:38:51.303 INFO:teuthology.orchestra.run.smithi196.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:38:51.305 INFO:teuthology.orchestra.run.smithi196.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:38:51.306 INFO:teuthology.orchestra.run.smithi196.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:38:51.307 INFO:teuthology.orchestra.run.smithi196.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:38:51.307 INFO:teuthology.orchestra.run.smithi196.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:38:51.312 INFO:teuthology.orchestra.run.smithi196.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:38:51.312 INFO:teuthology.orchestra.run.smithi196.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:38:51.318 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:38:51.323 INFO:teuthology.orchestra.run.smithi196.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:38:51.323 INFO:teuthology.orchestra.run.smithi196.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:38:51.324 INFO:teuthology.orchestra.run.smithi196.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:38:51.344 INFO:teuthology.orchestra.run.smithi196.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:38:51.346 INFO:teuthology.orchestra.run.smithi196.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:38:51.346 INFO:teuthology.orchestra.run.smithi196.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:38:51.353 INFO:teuthology.orchestra.run.smithi196.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi196.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi196.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:38:51.354 INFO:teuthology.orchestra.run.smithi129.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:38:51.355 INFO:teuthology.orchestra.run.smithi196.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:38:51.357 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:38:51.364 INFO:teuthology.orchestra.run.smithi196.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:38:51.367 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:38:51.406 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:38:51.437 INFO:teuthology.orchestra.run.smithi129.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:38:51.440 INFO:teuthology.orchestra.run.smithi196.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:38:51.442 INFO:teuthology.orchestra.run.smithi129.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:38:51.455 INFO:teuthology.orchestra.run.smithi129.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:38:51.459 INFO:teuthology.orchestra.run.smithi129.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:38:51.459 INFO:teuthology.orchestra.run.smithi129.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:38:51.467 INFO:teuthology.orchestra.run.smithi129.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:38:51.470 INFO:teuthology.orchestra.run.smithi196.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:38:51.470 INFO:teuthology.orchestra.run.smithi129.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:38:51.472 INFO:teuthology.orchestra.run.smithi129.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:38:51.473 INFO:teuthology.orchestra.run.smithi129.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:38:51.473 INFO:teuthology.orchestra.run.smithi129.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:38:51.481 INFO:teuthology.orchestra.run.smithi129.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:38:51.485 INFO:teuthology.orchestra.run.smithi129.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:38:51.487 INFO:teuthology.orchestra.run.smithi129.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:38:51.500 INFO:teuthology.orchestra.run.smithi196.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:38:51.503 INFO:teuthology.orchestra.run.smithi196.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:38:51.509 INFO:teuthology.orchestra.run.smithi129.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:38:51.509 INFO:teuthology.orchestra.run.smithi129.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:38:51.511 INFO:teuthology.orchestra.run.smithi129.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:38:51.511 INFO:teuthology.orchestra.run.smithi129.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:38:51.511 INFO:teuthology.orchestra.run.smithi129.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:38:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:38:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:38:51.512 INFO:teuthology.orchestra.run.smithi196.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:38:51.515 INFO:teuthology.orchestra.run.smithi196.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:38:51.527 INFO:teuthology.orchestra.run.smithi196.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:38:51.528 INFO:teuthology.orchestra.run.smithi129.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:38:51.529 INFO:teuthology.orchestra.run.smithi129.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:38:51.532 INFO:teuthology.orchestra.run.smithi129.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:38:51.548 INFO:teuthology.orchestra.run.smithi129.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:38:51.548 INFO:teuthology.orchestra.run.smithi129.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:38:51.549 INFO:teuthology.orchestra.run.smithi129.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:38:51.550 INFO:teuthology.orchestra.run.smithi129.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:38:51.550 INFO:teuthology.orchestra.run.smithi129.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:38:51.550 INFO:teuthology.orchestra.run.smithi129.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:38:51.552 INFO:teuthology.orchestra.run.smithi129.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:38:51.568 INFO:teuthology.orchestra.run.smithi129.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:38:51.568 INFO:teuthology.orchestra.run.smithi129.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:38:51.573 INFO:teuthology.orchestra.run.smithi129.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:38:51.587 INFO:teuthology.orchestra.run.smithi129.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:38:51.587 INFO:teuthology.orchestra.run.smithi129.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:38:51.588 INFO:teuthology.orchestra.run.smithi129.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:38:51.590 INFO:teuthology.orchestra.run.smithi129.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:38:51.592 INFO:teuthology.orchestra.run.smithi129.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:38:51.592 INFO:teuthology.orchestra.run.smithi129.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:38:51.594 INFO:teuthology.orchestra.run.smithi129.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:38:51.616 INFO:teuthology.orchestra.run.smithi129.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:38:51.619 INFO:teuthology.orchestra.run.smithi129.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:38:51.619 INFO:teuthology.orchestra.run.smithi129.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:38:51.636 INFO:teuthology.orchestra.run.smithi129.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:38:51.638 INFO:teuthology.orchestra.run.smithi129.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:38:51.693 INFO:teuthology.orchestra.run.smithi129.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:38:51.694 INFO:teuthology.orchestra.run.smithi129.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:38:51.695 INFO:teuthology.orchestra.run.smithi129.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:38:51.695 INFO:teuthology.orchestra.run.smithi129.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:38:51.696 INFO:teuthology.orchestra.run.smithi129.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:38:51.714 INFO:teuthology.orchestra.run.smithi129.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:38:51.716 INFO:teuthology.orchestra.run.smithi129.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:38:51.716 INFO:teuthology.orchestra.run.smithi129.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:38:51.735 INFO:teuthology.orchestra.run.smithi129.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:38:51.735 INFO:teuthology.orchestra.run.smithi129.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:38:51.736 INFO:teuthology.orchestra.run.smithi129.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:38:51.745 INFO:teuthology.orchestra.run.smithi129.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:38:51.745 INFO:teuthology.orchestra.run.smithi129.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:38:51.745 INFO:teuthology.orchestra.run.smithi129.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:38:51.746 INFO:teuthology.orchestra.run.smithi129.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:38:51.746 INFO:teuthology.orchestra.run.smithi129.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:38:51.750 INFO:teuthology.orchestra.run.smithi129.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:38:51.764 INFO:teuthology.orchestra.run.smithi129.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:38:51.766 INFO:teuthology.orchestra.run.smithi129.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:38:51.766 INFO:teuthology.orchestra.run.smithi129.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:38:51.770 INFO:teuthology.orchestra.run.smithi129.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:38:51.784 INFO:teuthology.orchestra.run.smithi129.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:38:51.784 INFO:teuthology.orchestra.run.smithi129.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:38:51.786 INFO:teuthology.orchestra.run.smithi129.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:38:51.786 INFO:teuthology.orchestra.run.smithi129.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:38:51.797 INFO:teuthology.orchestra.run.smithi129.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:38:51.799 INFO:teuthology.orchestra.run.smithi129.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:38:51.814 INFO:teuthology.orchestra.run.smithi129.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:38:51.814 INFO:teuthology.orchestra.run.smithi129.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:38:51.815 INFO:teuthology.orchestra.run.smithi129.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:38:51.815 INFO:teuthology.orchestra.run.smithi129.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:38:51.833 INFO:teuthology.orchestra.run.smithi129.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:38:51.834 INFO:teuthology.orchestra.run.smithi129.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:38:51.841 INFO:teuthology.orchestra.run.smithi129.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:38:51.841 INFO:teuthology.orchestra.run.smithi129.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:38:51.841 INFO:teuthology.orchestra.run.smithi129.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:38:51.843 INFO:teuthology.orchestra.run.smithi129.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:38:51.853 INFO:teuthology.orchestra.run.smithi129.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:38:51.855 INFO:teuthology.orchestra.run.smithi196.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:38:51.866 INFO:teuthology.orchestra.run.smithi196.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:38:51.882 INFO:teuthology.orchestra.run.smithi196.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:38:52.732 INFO:teuthology.orchestra.run.smithi129.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:38:53.337 INFO:teuthology.orchestra.run.smithi129.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:38:53.390 INFO:teuthology.orchestra.run.smithi196.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:38:53.409 INFO:teuthology.orchestra.run.smithi129.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:38:53.456 INFO:teuthology.orchestra.run.smithi129.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:38:53.458 INFO:teuthology.orchestra.run.smithi129.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:38:53.470 INFO:teuthology.orchestra.run.smithi129.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:38:53.475 INFO:teuthology.orchestra.run.smithi129.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:38:53.517 INFO:teuthology.orchestra.run.smithi129.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:38:53.562 INFO:teuthology.orchestra.run.smithi196.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:38:53.574 INFO:teuthology.orchestra.run.smithi196.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:38:53.578 INFO:teuthology.orchestra.run.smithi196.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:38:53.651 INFO:teuthology.orchestra.run.smithi196.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:38:53.961 INFO:teuthology.orchestra.run.smithi196.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:38:54.159 INFO:teuthology.orchestra.run.smithi129.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:38:54.166 INFO:teuthology.orchestra.run.smithi129.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:38:54.207 INFO:teuthology.orchestra.run.smithi129.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:38:55.395 INFO:teuthology.orchestra.run.smithi196.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:38:55.396 INFO:teuthology.orchestra.run.smithi196.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:38:55.413 INFO:teuthology.orchestra.run.smithi196.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:38:55.521 INFO:teuthology.orchestra.run.smithi196.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:38:55.564 INFO:teuthology.orchestra.run.smithi196.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:38:55.573 INFO:teuthology.orchestra.run.smithi196.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:38:55.720 INFO:teuthology.orchestra.run.smithi196.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:38:56.118 INFO:teuthology.orchestra.run.smithi129.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:38:56.251 INFO:teuthology.orchestra.run.smithi196.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:38:56.252 INFO:teuthology.orchestra.run.smithi196.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:38:56.380 INFO:teuthology.orchestra.run.smithi129.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:38:56.397 INFO:teuthology.orchestra.run.smithi129.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:38:56.403 INFO:teuthology.orchestra.run.smithi129.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:38:56.510 INFO:teuthology.orchestra.run.smithi129.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:38:56.936 INFO:teuthology.orchestra.run.smithi129.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:38:58.576 INFO:teuthology.orchestra.run.smithi129.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:38:58.577 INFO:teuthology.orchestra.run.smithi129.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:38:58.593 INFO:teuthology.orchestra.run.smithi129.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:38:58.708 INFO:teuthology.orchestra.run.smithi129.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:38:58.758 INFO:teuthology.orchestra.run.smithi129.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:38:58.769 INFO:teuthology.orchestra.run.smithi129.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:38:58.922 INFO:teuthology.orchestra.run.smithi129.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:38:59.494 INFO:teuthology.orchestra.run.smithi129.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:38:59.494 INFO:teuthology.orchestra.run.smithi129.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:38:59.752 INFO:teuthology.orchestra.run.smithi196.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:38:59.755 INFO:teuthology.orchestra.run.smithi196.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:38:59.760 INFO:teuthology.orchestra.run.smithi196.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:38:59.761 INFO:teuthology.orchestra.run.smithi196.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:38:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:38:59.763 INFO:teuthology.orchestra.run.smithi196.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:39:00.489 INFO:teuthology.orchestra.run.smithi196.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:39:01.099 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 180 MB in 10s (18.4 MB/s) 2024-04-25T00:39:01.236 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:39:01.331 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T00:39:01.337 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:39:01.375 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:39:01.670 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:39:01.697 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:39:01.728 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:39:01.876 INFO:teuthology.orchestra.run.smithi129.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:39:01.878 INFO:teuthology.orchestra.run.smithi129.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:39:01.883 INFO:teuthology.orchestra.run.smithi129.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:39:01.884 INFO:teuthology.orchestra.run.smithi129.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:39:01.885 INFO:teuthology.orchestra.run.smithi129.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:39:01.886 INFO:teuthology.orchestra.run.smithi129.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:39:02.040 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:39:02.065 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:39:02.106 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:39:02.218 INFO:teuthology.orchestra.run.smithi129.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:39:02.484 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:39:02.510 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:39:02.592 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:39:02.833 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 180 MB in 11s (16.2 MB/s) 2024-04-25T00:39:02.945 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:39:02.946 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:39:02.973 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:39:03.002 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T00:39:03.007 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:39:03.012 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:39:03.035 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:39:03.314 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:39:03.340 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:39:03.344 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:39:03.373 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:39:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:39:03.549 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:39:03.733 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:39:03.760 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:39:03.800 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:39:03.977 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:39:04.003 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:39:04.052 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:39:04.161 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:39:04.188 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:39:04.270 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:39:04.464 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:39:04.490 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:39:04.530 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:39:04.582 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:39:04.609 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:39:04.640 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:39:04.883 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:39:04.909 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:39:04.943 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:39:04.949 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:39:04.970 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:39:05.010 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:39:05.278 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:39:05.304 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:39:05.336 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:39:05.455 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:39:05.482 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:39:05.530 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:39:05.680 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librados2. 2024-04-25T00:39:05.707 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:05.747 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:05.892 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:39:05.918 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:39:05.950 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:39:06.200 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:39:06.226 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:06.266 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:06.278 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:39:06.304 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:39:06.345 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:39:06.673 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:39:06.688 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:39:06.736 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:39:06.748 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:39:06.762 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:06.802 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:07.034 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librados2. 2024-04-25T00:39:07.055 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:39:07.061 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:07.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:07.101 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:07.113 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:07.441 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:39:07.468 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:07.507 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:07.571 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:39:07.597 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:07.637 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:07.760 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:39:07.775 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:07.818 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:08.138 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:39:08.164 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:08.204 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:08.240 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:39:08.266 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:08.307 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:08.543 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:39:08.558 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:08.601 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:08.666 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:39:08.692 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:39:08.849 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:39:08.929 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:39:08.955 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:08.996 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:09.248 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:39:09.268 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:39:09.271 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:09.294 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:09.315 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:09.335 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:09.601 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:39:09.627 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:09.668 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:09.688 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:39:09.714 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:39:09.754 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:39:09.954 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:39:09.969 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:39:10.004 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:39:10.032 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:39:10.047 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:39:10.090 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:39:10.298 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:39:10.314 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:10.357 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:10.393 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:39:10.419 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:39:10.451 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:39:10.710 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:39:10.736 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:39:10.777 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:39:10.804 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:39:10.831 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:39:10.871 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:39:11.054 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:39:11.070 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:39:11.113 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:39:11.191 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:39:11.217 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:39:11.257 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:39:11.374 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:39:11.389 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:39:11.432 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:39:11.476 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package zip. 2024-04-25T00:39:11.491 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:39:11.526 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:39:11.778 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:39:11.804 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:39:11.813 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package unzip. 2024-04-25T00:39:11.839 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:39:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:39:11.879 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:39:12.156 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:39:12.182 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:39:12.222 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:39:12.258 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:39:12.284 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:39:12.324 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:39:12.425 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package zip. 2024-04-25T00:39:12.440 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:39:12.483 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:39:12.657 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:39:12.683 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:12.715 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:12.803 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package unzip. 2024-04-25T00:39:12.830 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:39:12.870 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:39:13.240 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:39:13.243 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:39:13.267 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:39:13.269 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:13.307 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:39:13.309 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:13.662 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:39:13.689 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:39:13.729 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:39:13.824 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:39:13.851 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:13.890 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:14.015 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:39:14.042 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:14.073 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:14.435 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:39:14.461 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:14.485 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:39:14.501 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:14.511 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:14.551 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:14.771 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:39:14.798 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:39:14.837 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:39:15.149 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:39:15.176 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:15.215 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:15.677 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:39:15.704 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:15.744 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:15.890 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:39:15.917 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:15.966 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:16.544 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:39:16.570 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:39:16.610 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:39:16.871 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:39:16.887 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:39:16.921 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:39:17.016 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:39:17.042 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:17.083 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:17.174 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:39:17.201 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:39:17.241 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:39:17.527 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:39:17.554 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:39:17.593 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:39:17.637 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:39:17.663 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:39:17.703 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:39:17.873 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:39:17.900 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:39:17.932 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:39:18.006 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:39:18.033 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:39:18.073 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:39:18.193 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:39:18.219 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:39:18.259 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:39:18.401 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:39:18.416 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:39:18.451 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:39:18.562 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:39:18.588 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:39:18.628 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:39:18.729 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:39:18.755 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:39:18.796 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:39:19.098 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:39:19.125 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:39:19.165 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:39:19.173 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:39:19.199 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:39:19.239 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:39:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:39:19.469 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:39:19.500 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:39:19.509 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:39:19.527 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:39:19.559 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:39:19.737 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:39:19.753 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:39:19.796 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:39:19.904 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:39:19.930 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:39:19.970 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:39:20.099 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:39:20.126 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:39:20.165 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:39:20.265 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:39:20.291 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:39:20.332 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:39:20.469 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:39:20.495 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:39:20.535 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:39:20.567 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:39:20.583 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:39:20.626 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:39:20.881 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:39:20.907 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:39:20.928 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:39:20.947 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:39:20.955 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:39:20.995 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:39:21.259 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:39:21.286 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:39:21.332 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:39:21.342 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:39:21.358 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:39:21.398 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:39:21.603 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:39:21.630 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:39:21.662 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:39:21.701 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:39:21.727 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:39:21.767 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:39:21.965 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:39:21.991 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:39:22.003 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:39:22.019 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:39:22.031 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:39:22.053 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:39:22.351 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:39:22.378 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:39:22.406 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:39:22.417 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:39:22.433 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:39:22.476 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:39:22.723 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:39:22.749 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:39:22.789 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:39:22.817 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:39:22.844 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:39:22.884 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:39:23.084 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:39:23.111 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:39:23.150 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:39:23.187 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:39:23.213 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:39:23.253 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:39:23.487 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:39:23.513 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:39:23.556 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:39:23.556 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:39:23.571 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:39:23.614 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:39:23.891 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:39:23.917 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:39:23.918 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:39:23.943 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:39:23.984 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:39:24.072 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:39:25.129 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:39:25.155 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:39:25.196 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:39:25.235 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:39:25.262 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:39:25.310 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:39:25.490 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:39:25.517 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:39:25.557 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:39:25.638 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:39:25.665 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:39:25.713 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:39:25.895 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:39:25.921 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:39:25.961 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:39:25.982 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:39:26.009 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:39:26.040 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:39:26.281 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:39:26.302 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:39:26.307 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:39:26.328 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:39:26.347 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:39:26.368 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:39:26.667 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:39:26.694 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:39:26.697 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:39:26.724 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:39:26.733 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:39:26.762 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:39:27.087 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:39:27.114 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:39:27.117 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:39:27.144 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:39:27.153 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:39:27.184 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:39:27.439 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:39:27.467 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:39:27.506 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:39:27.521 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:39:27.548 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:39:27.586 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:39:27.801 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:39:27.827 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:27.865 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:39:27.867 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:27.880 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:39:27.914 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:39:28.268 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:39:28.280 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:39:28.295 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:39:28.307 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:28.334 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:39:28.346 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:28.595 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:39:28.622 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:39:28.653 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:39:28.674 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:39:28.701 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:39:28.741 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:39:28.924 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:39:28.950 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:28.990 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:29.044 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:39:29.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:39:29.110 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:39:29.386 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:39:29.414 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:29.453 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:29.638 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:39:29.664 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:29.704 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:29.772 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:39:29.800 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:39:29.847 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:39:30.066 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:39:30.094 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:30.124 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:30.133 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:39:30.161 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:39:30.191 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:39:30.503 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:39:30.530 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:30.569 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:30.694 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:39:30.721 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:30.760 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:30.973 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:39:31.000 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:31.039 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:31.626 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:39:31.653 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:31.692 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:31.839 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph. 2024-04-25T00:39:31.865 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:31.905 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:32.233 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:39:32.259 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:32.299 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:32.652 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:39:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:32.710 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:32.729 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph. 2024-04-25T00:39:32.756 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:32.795 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:33.080 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:39:33.106 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:33.123 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:39:33.146 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:33.150 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:33.190 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:33.474 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:39:33.501 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:39:33.540 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:39:33.576 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:39:33.602 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:33.642 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:33.843 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:39:33.858 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:39:33.901 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:39:34.003 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:39:34.019 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:34.053 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:34.581 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:39:34.608 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:39:34.648 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:39:34.948 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:39:34.967 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:39:34.975 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:39:34.994 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:39:35.014 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:39:35.025 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:39:35.384 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:39:35.411 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:39:35.450 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:39:35.803 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:39:35.830 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:35.869 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:36.055 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:39:36.083 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:39:36.122 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:39:36.198 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:39:36.225 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:39:36.273 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:39:36.425 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:39:36.452 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:39:36.483 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:39:36.575 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:39:36.603 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:39:36.642 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:39:36.819 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:39:36.846 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:36.877 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:39:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:37.053 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:37.198 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:39:37.213 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:39:37.247 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:39:37.500 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:39:37.528 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:39:37.566 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:39:37.886 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:39:37.902 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:37.936 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:38.258 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:39:38.274 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:39:38.316 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:39:38.828 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:39:38.855 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:39:38.897 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:39:39.175 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:39:39.202 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:39:39.241 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:39:39.273 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:39:39.300 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:39:39.424 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:39:39.728 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:39:39.736 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:39:39.743 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:39:39.763 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:39:39.777 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:39:39.802 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:39:40.157 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:39:40.183 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:39:40.308 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:39:40.528 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:39:40.555 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:40.586 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:39:40.594 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:40.613 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:39:40.644 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:39:41.345 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:39:41.373 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:41.411 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:41.572 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:39:41.599 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:39:41.638 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:39:41.941 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:39:41.968 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:39:42.008 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:39:42.320 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:39:42.346 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:39:42.365 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:39:42.377 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:39:42.392 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:39:42.431 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:39:42.664 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:39:42.679 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:39:42.713 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:39:42.717 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:39:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:39:42.784 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:39:43.026 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:39:43.053 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:39:43.092 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:39:43.112 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:39:43.139 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:39:43.179 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:39:43.412 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:39:43.439 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:39:43.478 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:39:43.532 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:39:43.559 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:39:43.598 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:39:43.798 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:39:43.825 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:39:43.864 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:39:43.986 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:39:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:39:44.060 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:39:44.159 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:39:44.183 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:39:44.217 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:39:44.414 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:39:44.441 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:39:44.480 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:39:44.908 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:39:44.924 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:39:44.958 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:39:45.098 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:39:45.126 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:45.164 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:45.219 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:39:45.249 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:39:45.286 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:39:45.492 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:39:45.519 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:39:45.558 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:39:45.895 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:39:45.922 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:39:45.969 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:39:45.983 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:39:46.011 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:46.050 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:46.272 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package jq. 2024-04-25T00:39:46.299 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:39:46.347 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:39:46.386 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:39:46.413 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:39:46.452 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:39:46.650 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package socat. 2024-04-25T00:39:46.665 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:39:46.716 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:39:46.839 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:39:46.866 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:39:46.913 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:39:47.036 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:39:47.052 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:39:47.094 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:39:47.216 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package jq. 2024-04-25T00:39:47.232 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:39:47.275 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:39:47.515 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:39:47.527 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package socat. 2024-04-25T00:39:47.543 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:39:47.543 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:47.585 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:39:47.590 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:47.880 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:39:47.896 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:39:47.938 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:39:48.294 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:39:48.320 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:48.359 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:49.885 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:39:49.913 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:49.952 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:50.490 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:39:50.518 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:50.556 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:50.655 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:39:50.682 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:50.721 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:50.867 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:39:50.895 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:39:50.934 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:39:51.184 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:39:51.195 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:39:51.212 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:51.219 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:51.259 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:51.261 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:51.547 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:39:51.563 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:51.587 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:39:51.605 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:51.614 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:39:51.653 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:39:51.841 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:39:51.869 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:51.899 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:51.914 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:39:51.930 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:51.981 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:52.200 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:39:52.202 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:39:52.216 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:39:52.230 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:39:52.258 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:52.268 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:39:52.502 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:39:52.518 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:52.560 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:52.632 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:39:52.659 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:39:52.698 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:39:52.888 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:39:52.916 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:39:52.955 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:39:53.018 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:39:53.046 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:39:53.084 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:39:53.285 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:39:53.314 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:39:53.359 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:39:53.490 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:39:53.518 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:39:53.556 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:39:53.705 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:39:53.733 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:39:53.771 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:39:53.792 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:39:53.808 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:39:53.850 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:39:54.235 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:39:54.262 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:39:54.295 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:39:54.309 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:39:54.323 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:39:54.361 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:39:54.629 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:39:54.645 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:39:54.664 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:39:54.692 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:39:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:39:54.739 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:39:55.042 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:39:55.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:39:55.109 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:39:55.174 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:39:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:39:55.240 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:39:55.370 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:39:55.397 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:39:55.428 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:39:55.485 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:39:55.501 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:39:55.534 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:39:55.781 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:39:55.809 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:39:55.813 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:39:55.841 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:39:55.850 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:39:55.879 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:39:56.169 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:39:56.197 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:39:56.199 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:39:56.227 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:39:56.235 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:39:56.266 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:39:56.594 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:39:56.622 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:39:56.676 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:39:56.715 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:39:56.742 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:39:56.772 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:39:56.997 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:39:57.024 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:39:57.076 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:39:57.079 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:39:57.103 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:39:57.142 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:39:57.428 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:39:57.455 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:39:57.494 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:39:57.551 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:39:57.579 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:39:57.617 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:39:57.848 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:39:57.875 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:39:57.914 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:39:57.928 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:39:57.944 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:39:57.986 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:39:58.183 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:39:58.200 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:39:58.241 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:39:58.255 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:39:58.284 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:39:58.313 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:39:58.667 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:39:58.695 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:39:58.734 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:39:59.020 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:39:59.036 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:39:59.054 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:39:59.078 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:39:59.081 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:59.120 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:59.807 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:39:59.835 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:39:59.873 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:39:59.940 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:39:59.968 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:40:00.006 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:00.310 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:40:00.578 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:40:00.668 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:40:00.684 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:40:00.704 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:40:00.762 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:00.977 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:40:01.222 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:40:01.293 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:40:01.473 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:40:01.599 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:40:01.703 INFO:teuthology.orchestra.run.smithi196.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:40:01.865 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:40:01.998 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:02.194 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:40:02.464 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user cephadm....done 2024-04-25T00:40:02.604 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:40:02.609 INFO:teuthology.orchestra.run.smithi129.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:40:02.676 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:02.872 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:40:03.123 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:40:03.233 INFO:teuthology.orchestra.run.smithi129.stdout:Adding system user cephadm....done 2024-04-25T00:40:03.249 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:40:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:40:03.508 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:40:03.652 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:40:03.759 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:40:03.860 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:40:03.878 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:40:03.979 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:40:04.154 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:40:04.221 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:40:04.473 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:40:04.505 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:40:04.599 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:40:04.757 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:40:04.909 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:40:05.008 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:40:05.259 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:40:05.269 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:40:05.494 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:40:05.537 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:40:05.780 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:40:05.995 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:40:06.047 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:40:06.262 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:40:06.299 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:40:06.530 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:06.558 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:40:06.798 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:40:06.851 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:40:06.924 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:40:07.025 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:40:07.094 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:07.259 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:40:07.360 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:40:07.503 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:40:07.594 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:40:07.604 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:40:07.730 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:40:07.968 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:40:07.989 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:40:08.018 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:40:08.072 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:40:08.115 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:40:08.198 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:40:08.375 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:40:08.567 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:40:08.684 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:40:08.746 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:40:08.772 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:40:08.828 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:40:08.927 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:40:08.929 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:40:09.169 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:40:09.272 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:40:09.398 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:40:09.479 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:40:09.605 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:40:09.665 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:40:09.933 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:40:10.276 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:40:10.357 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:40:10.377 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:40:10.608 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:40:10.725 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:40:10.826 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:40:11.045 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:40:11.111 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:40:11.296 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:40:11.422 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:40:11.512 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:40:11.548 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:40:11.763 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:40:11.849 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:40:12.148 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:40:12.234 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:40:12.457 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:40:12.485 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:40:12.758 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:40:12.853 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:40:13.101 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:40:13.146 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:40:13.336 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:40:13.439 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:40:13.462 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:13.780 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:40:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:40:14.041 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:40:14.097 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:40:14.151 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:14.435 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:40:14.744 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:40:15.311 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:40:15.563 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:40:15.697 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:40:15.942 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:40:15.956 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:40:16.107 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:40:16.225 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:40:16.385 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:40:16.510 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:40:16.710 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:40:16.744 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:40:16.804 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:40:16.930 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:40:16.995 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:40:17.055 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:40:17.121 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:40:17.340 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:40:17.430 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:40:17.599 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:40:17.681 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:40:17.842 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:40:17.941 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:40:17.951 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:40:18.184 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:40:18.253 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:40:18.426 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:40:18.504 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:40:18.694 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:40:18.763 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:40:18.820 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:40:19.022 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:40:19.104 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:40:19.230 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:40:19.282 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:40:19.523 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:40:19.541 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:40:19.642 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:40:19.808 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:40:19.910 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:40:20.045 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:40:20.058 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:40:20.184 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:40:20.286 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:40:20.329 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:40:20.375 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:40:20.613 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:40:20.630 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:40:20.738 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:40:20.889 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:40:21.015 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:40:21.114 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:40:21.124 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:40:21.196 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:40:21.435 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:40:21.544 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:40:21.675 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:40:21.783 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:40:21.904 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:40:22.017 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:40:22.144 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:40:22.163 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:40:22.289 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:40:22.453 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:40:22.540 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:40:22.666 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:40:22.762 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:40:22.958 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:40:23.030 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:40:23.243 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:40:23.250 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:40:23.509 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:40:23.527 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:40:23.739 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:40:23.777 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:40:23.998 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:40:24.037 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:40:24.258 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:40:24.413 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:40:24.526 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:40:24.690 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:40:24.919 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:40:25.017 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:40:25.143 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:25.204 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:40:25.286 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:25.429 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:40:25.539 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:40:25.666 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:25.765 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:25.766 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:25.876 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:40:25.899 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.008 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:40:26.229 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.268 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.355 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.386 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.481 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:40:26.525 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.742 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:40:26.742 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:40:26.757 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:26.883 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.001 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.203 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:40:27.204 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:40:27.204 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.330 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.465 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.591 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.702 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.708 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:40:27.820 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:27.946 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:28.072 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:28.198 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:40:28.912 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.038 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.164 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.298 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.424 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.452 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.550 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.578 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.704 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.829 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:29.980 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:30.017 INFO:teuthology.orchestra.run.smithi196.stdout:Adding group ceph....done 2024-04-25T00:40:30.107 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:30.394 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user ceph....done 2024-04-25T00:40:30.554 INFO:teuthology.orchestra.run.smithi196.stdout:Setting system user ceph properties....done 2024-04-25T00:40:30.567 INFO:teuthology.orchestra.run.smithi196.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:40:30.625 INFO:teuthology.orchestra.run.smithi129.stdout:Adding group ceph....done 2024-04-25T00:40:30.703 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:40:30.983 INFO:teuthology.orchestra.run.smithi129.stdout:Adding system user ceph....done 2024-04-25T00:40:31.020 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:40:31.143 INFO:teuthology.orchestra.run.smithi129.stdout:Setting system user ceph properties....done 2024-04-25T00:40:31.161 INFO:teuthology.orchestra.run.smithi129.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:40:31.300 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:40:31.476 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:31.632 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:40:31.664 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:40:31.664 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:40:32.090 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:32.144 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:32.270 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:32.285 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:40:32.285 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:40:32.721 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:40:32.721 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:40:32.926 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:33.027 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:33.163 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:33.455 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:40:33.456 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:40:33.634 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:40:33.904 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:34.099 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:34.253 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:40:34.329 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:40:34.330 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:40:34.738 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:34.784 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:34.961 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:40:34.961 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:40:34.999 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:40:35.000 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:40:35.423 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:35.435 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:35.650 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:40:35.650 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:40:35.680 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:40:35.680 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:40:36.100 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:36.153 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:36.271 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:36.357 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:40:36.357 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:40:36.413 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:36.643 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:40:36.643 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:40:36.869 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:36.995 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:37.146 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:37.296 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:37.363 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:40:37.363 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:40:37.427 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:37.528 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:37.671 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:37.851 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:38.006 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:38.032 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:40:38.150 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:40:38.248 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:38.285 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:40:38.458 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:40:38.795 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:40:38.895 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:40:39.048 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:40:39.068 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:40:39.689 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:40:39.692 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:40:39.792 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T00:40:39.993 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T00:40:39.993 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T00:40:40.098 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:40:40.182 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:40:40.183 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:40:40.183 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:40:40.183 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-25T00:40:40.183 INFO:teuthology.orchestra.run.smithi196.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:40:40.183 INFO:teuthology.orchestra.run.smithi196.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:40:40.184 INFO:teuthology.orchestra.run.smithi196.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:40:40.184 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:40:40.184 INFO:teuthology.orchestra.run.smithi196.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:40:40.185 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-25T00:40:40.185 INFO:teuthology.orchestra.run.smithi196.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:40:40.185 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:40:40.185 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:40:40.224 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-25T00:40:40.225 INFO:teuthology.orchestra.run.smithi196.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:40:40.225 INFO:teuthology.orchestra.run.smithi196.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:40:40.226 INFO:teuthology.orchestra.run.smithi196.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:40:40.226 INFO:teuthology.orchestra.run.smithi196.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:40:40.226 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:40:40.226 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:40:40.226 INFO:teuthology.orchestra.run.smithi196.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:40:40.227 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-04-25T00:40:40.227 INFO:teuthology.orchestra.run.smithi196.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:40:40.408 INFO:teuthology.orchestra.run.smithi196.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T00:40:40.408 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:40:40.409 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:40:40.409 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:40:40.741 INFO:teuthology.orchestra.run.smithi129.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:40:40.744 DEBUG:teuthology.orchestra.run.smithi129:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:40:40.844 INFO:teuthology.orchestra.run.smithi129.stdout:Reading package lists... 2024-04-25T00:40:41.029 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:40:41.043 INFO:teuthology.orchestra.run.smithi129.stdout:Building dependency tree... 2024-04-25T00:40:41.044 INFO:teuthology.orchestra.run.smithi129.stdout:Reading state information... 2024-04-25T00:40:41.072 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:40:41.209 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:40:41.210 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:40:41.211 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:40:41.230 INFO:teuthology.orchestra.run.smithi129.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:40:41.231 INFO:teuthology.orchestra.run.smithi129.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:40:41.232 INFO:teuthology.orchestra.run.smithi129.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:40:41.232 INFO:teuthology.orchestra.run.smithi129.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:40:41.232 INFO:teuthology.orchestra.run.smithi129.stdout:The following additional packages will be installed: 2024-04-25T00:40:41.232 INFO:teuthology.orchestra.run.smithi129.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:40:41.232 INFO:teuthology.orchestra.run.smithi129.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:40:41.233 INFO:teuthology.orchestra.run.smithi129.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:40:41.233 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:40:41.233 INFO:teuthology.orchestra.run.smithi129.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:40:41.234 INFO:teuthology.orchestra.run.smithi129.stdout:Suggested packages: 2024-04-25T00:40:41.234 INFO:teuthology.orchestra.run.smithi129.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:40:41.234 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:40:41.234 INFO:teuthology.orchestra.run.smithi129.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:40:41.239 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:40:41.241 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:40:41.245 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:40:41.248 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:40:41.249 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:40:41.274 INFO:teuthology.orchestra.run.smithi129.stdout:The following NEW packages will be installed: 2024-04-25T00:40:41.274 INFO:teuthology.orchestra.run.smithi129.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:40:41.274 INFO:teuthology.orchestra.run.smithi129.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:40:41.275 INFO:teuthology.orchestra.run.smithi129.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:40:41.275 INFO:teuthology.orchestra.run.smithi129.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:40:41.275 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:40:41.275 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:40:41.276 INFO:teuthology.orchestra.run.smithi129.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:40:41.276 INFO:teuthology.orchestra.run.smithi129.stdout:The following packages will be upgraded: 2024-04-25T00:40:41.277 INFO:teuthology.orchestra.run.smithi129.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:40:41.415 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:40:41.458 INFO:teuthology.orchestra.run.smithi129.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T00:40:41.458 INFO:teuthology.orchestra.run.smithi129.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:40:41.459 INFO:teuthology.orchestra.run.smithi129.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:40:41.459 INFO:teuthology.orchestra.run.smithi129.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:40:41.709 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:40:41.718 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:40:41.722 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:40:41.807 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:40:42.086 INFO:teuthology.orchestra.run.smithi129.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:40:42.103 INFO:teuthology.orchestra.run.smithi129.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:40:42.228 INFO:teuthology.orchestra.run.smithi129.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:40:42.229 INFO:teuthology.orchestra.run.smithi129.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:40:42.230 INFO:teuthology.orchestra.run.smithi129.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:40:42.258 INFO:teuthology.orchestra.run.smithi129.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:40:42.260 INFO:teuthology.orchestra.run.smithi129.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:40:42.264 INFO:teuthology.orchestra.run.smithi129.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:40:42.266 INFO:teuthology.orchestra.run.smithi129.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:40:42.267 INFO:teuthology.orchestra.run.smithi129.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:40:42.433 INFO:teuthology.orchestra.run.smithi129.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:40:42.727 INFO:teuthology.orchestra.run.smithi129.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:40:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:40:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:40:42.826 INFO:teuthology.orchestra.run.smithi129.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:40:43.928 INFO:teuthology.orchestra.run.smithi129.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:40:43.932 INFO:teuthology.orchestra.run.smithi129.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:40:43.933 INFO:teuthology.orchestra.run.smithi129.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:40:43.939 INFO:teuthology.orchestra.run.smithi129.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:40:43.940 INFO:teuthology.orchestra.run.smithi129.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:40:43.951 INFO:teuthology.orchestra.run.smithi129.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:40:44.118 INFO:teuthology.orchestra.run.smithi129.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:40:44.714 INFO:teuthology.orchestra.run.smithi129.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:40:44.714 INFO:teuthology.orchestra.run.smithi129.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:40:44.893 INFO:teuthology.orchestra.run.smithi129.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:40:44.894 INFO:teuthology.orchestra.run.smithi129.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:40:44.932 INFO:teuthology.orchestra.run.smithi129.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:40:44.994 INFO:teuthology.orchestra.run.smithi129.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:40:45.000 INFO:teuthology.orchestra.run.smithi129.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:40:45.002 INFO:teuthology.orchestra.run.smithi129.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:40:45.003 INFO:teuthology.orchestra.run.smithi129.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:40:45.003 INFO:teuthology.orchestra.run.smithi129.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:40:45.209 INFO:teuthology.orchestra.run.smithi129.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:40:45.619 INFO:teuthology.orchestra.run.smithi129.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:40:46.368 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:40:46.373 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:40:46.373 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:40:46.379 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:40:46.380 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:40:46.390 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:40:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:40:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:40:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:40:46.418 INFO:teuthology.orchestra.run.smithi129.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:40:46.419 INFO:teuthology.orchestra.run.smithi129.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:40:46.419 INFO:teuthology.orchestra.run.smithi129.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:40:46.444 INFO:teuthology.orchestra.run.smithi129.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:40:46.445 INFO:teuthology.orchestra.run.smithi129.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:40:46.558 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:40:46.853 INFO:teuthology.orchestra.run.smithi129.stdout:Preconfiguring packages ... 2024-04-25T00:40:47.167 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:40:47.167 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:40:47.344 INFO:teuthology.orchestra.run.smithi129.stdout:Fetched 55.7 MB in 5s (10.8 MB/s) 2024-04-25T00:40:47.347 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:40:47.348 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:40:47.386 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:40:47.448 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:40:47.455 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:40:47.456 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:40:47.457 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:40:47.459 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:40:47.668 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:40:47.670 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:40:47.676 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:40:47.831 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:40:48.098 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:40:48.468 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:40:49.405 INFO:teuthology.orchestra.run.smithi129.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:40:49.661 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package flex. 2024-04-25T00:40:49.766 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:40:49.768 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:40:49.817 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:40:50.366 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:40:50.539 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:40:50.975 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:40:51.503 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:40:51.508 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:40:51.638 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:40:52.055 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:40:52.170 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:40:52.548 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:40:52.921 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package bison. 2024-04-25T00:40:53.025 INFO:teuthology.orchestra.run.smithi129.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:40:53.031 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:40:53.081 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:40:53.524 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:40:53.551 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:40:53.590 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:40:53.902 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package dump. 2024-04-25T00:40:53.929 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:40:53.960 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:40:54.313 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package indent. 2024-04-25T00:40:54.330 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:40:54.372 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:40:54.632 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:40:54.648 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:40:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:40:55.011 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:40:55.039 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:40:55.078 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:40:55.297 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:40:55.313 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:40:55.347 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:40:55.692 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:40:55.720 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:40:55.759 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:40:56.013 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:40:56.029 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:40:56.071 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:40:56.274 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:40:56.290 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:40:56.332 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:40:56.660 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:40:56.688 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:40:56.727 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:40:58.406 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:40:58.434 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:40:58.472 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:40:58.700 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:40:58.716 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:40:58.750 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:40:59.069 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:40:59.097 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:40:59.136 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:40:59.480 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:40:59.508 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:40:59.547 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:40:59.892 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:40:59.919 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:40:59.958 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:41:00.253 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:41:00.280 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:00.319 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:00.672 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:41:00.700 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:00.739 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:01.041 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:41:01.069 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:01.108 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:01.327 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:41:01.351 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:41:01.385 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:41:01.889 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:41:01.916 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:41:01.955 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:41:02.308 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:41:02.336 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:41:02.375 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:03.099 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:41:03.127 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:41:03.174 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:41:03.840 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package ppp. 2024-04-25T00:41:03.868 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:41:04.160 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:41:04.469 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:41:04.498 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:41:04.536 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:41:04.789 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:41:04.805 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:41:04.847 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:41:05.117 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:41:05.145 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:41:05.200 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:41:05.487 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:41:05.515 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:05.554 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:06.150 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:41:06.178 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:41:06.235 INFO:teuthology.orchestra.run.smithi129.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:41:06.264 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:41:06.757 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:41:06.785 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:06.826 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:08.176 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:41:08.192 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:41:08.226 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:41:08.470 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:41:08.486 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:41:08.528 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:41:08.780 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:41:08.808 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:41:08.847 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:41:09.166 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:41:09.194 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:41:09.232 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:41:09.510 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package tcl. 2024-04-25T00:41:09.526 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:41:09.572 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:41:09.938 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:41:09.966 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:41:10.004 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:41:10.325 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:41:10.341 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:41:10.360 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:41:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:41:10.361 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:41:10.362 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:41:10.375 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:41:10.396 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:41:10.396 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:41:10.753 INFO:teuthology.orchestra.run.smithi129.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:41:10.781 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:41:10.828 INFO:teuthology.orchestra.run.smithi129.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:41:11.124 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:41:11.334 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:41:11.460 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:41:11.586 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:11.712 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:41:11.838 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:41:11.947 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:12.426 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:41:12.456 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:41:12.518 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:41:12.519 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:41:12.884 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:41:12.884 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:41:12.960 INFO:teuthology.orchestra.run.smithi196.stdout:Preconfiguring packages ... 2024-04-25T00:41:13.183 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 55.7 MB in 32s (1,724 kB/s) 2024-04-25T00:41:13.405 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:41:13.820 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:41:13.825 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:41:13.957 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:41:13.970 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:41:14.083 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:41:14.209 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:41:14.335 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:41:14.461 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:41:14.566 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:41:14.587 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:41:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:41:15.512 INFO:teuthology.orchestra.run.smithi196.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:41:15.560 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:41:15.686 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:41:15.780 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:41:15.808 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package flex. 2024-04-25T00:41:15.854 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:15.914 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:41:15.920 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:41:15.975 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:41:16.006 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:41:16.157 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:41:16.299 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:41:16.425 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:41:16.500 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:41:16.540 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:41:16.610 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:41:16.673 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:41:16.736 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:41:16.912 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:41:17.064 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:41:17.117 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:41:17.214 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:41:17.416 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:41:17.542 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:41:17.661 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:41:17.666 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:41:17.797 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:41:18.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:18.003 INFO:teuthology.orchestra.run.smithi129.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:41:18.003 INFO:teuthology.orchestra.run.smithi129.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:41:18.003 INFO:teuthology.orchestra.run.smithi129.stdout:- enp3s0f1 2024-04-25T00:41:18.003 INFO:teuthology.orchestra.run.smithi129.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:41:18.003 INFO:teuthology.orchestra.run.smithi129.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:41:18.004 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:18.110 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:41:18.188 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:41:18.328 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:41:18.419 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:41:18.655 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:41:18.716 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:41:19.070 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package bison. 2024-04-25T00:41:19.176 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:41:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:41:19.239 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:41:19.411 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:41:19.546 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:41:19.689 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:41:19.715 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:41:19.743 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:41:19.790 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:41:19.865 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:41:20.234 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package dump. 2024-04-25T00:41:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:41:20.301 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:41:20.588 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package indent. 2024-04-25T00:41:20.604 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:20.655 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:41:20.734 INFO:teuthology.orchestra.run.smithi129.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:41:20.957 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:41:20.971 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:20.971 INFO:teuthology.orchestra.run.smithi129.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:41:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:41:21.032 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:41:21.135 INFO:teuthology.orchestra.run.smithi129.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:41:21.136 INFO:teuthology.orchestra.run.smithi129.stdout:Removing obsolete dictionary files: 2024-04-25T00:41:21.411 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:41:21.439 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:41:21.478 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:41:21.680 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:41:21.772 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:41:21.800 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:41:21.839 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:41:22.057 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:22.134 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:41:22.161 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:41:22.200 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:41:22.389 INFO:teuthology.orchestra.run.smithi129.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:41:22.408 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout:This user must also own the server process. 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout:The default text search configuration will be set to "english". 2024-04-25T00:41:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:22.453 INFO:teuthology.orchestra.run.smithi129.stdout:Data page checksums are disabled. 2024-04-25T00:41:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:22.453 INFO:teuthology.orchestra.run.smithi129.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:41:22.453 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:41:22.454 INFO:teuthology.orchestra.run.smithi129.stdout:creating subdirectories ... ok 2024-04-25T00:41:22.454 INFO:teuthology.orchestra.run.smithi129.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:41:22.468 INFO:teuthology.orchestra.run.smithi129.stdout:selecting default max_connections ... 100 2024-04-25T00:41:22.481 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:41:22.489 INFO:teuthology.orchestra.run.smithi129.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:41:22.499 INFO:teuthology.orchestra.run.smithi129.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:41:22.500 INFO:teuthology.orchestra.run.smithi129.stdout:creating configuration files ... ok 2024-04-25T00:41:22.520 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:41:22.616 INFO:teuthology.orchestra.run.smithi129.stdout:running bootstrap script ... ok 2024-04-25T00:41:22.772 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:41:22.788 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:41:22.830 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:41:23.125 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:41:23.153 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:41:23.184 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:41:23.308 INFO:teuthology.orchestra.run.smithi129.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:41:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:syncing data to disk ... ok 2024-04-25T00:41:24.342 INFO:teuthology.orchestra.run.smithi129.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:41:24.803 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:41:24.831 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:41:24.870 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:41:25.152 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:41:25.181 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:41:25.231 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:41:25.558 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:41:25.586 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:41:25.625 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:25.936 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:41:25.964 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:41:26.003 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:41:26.331 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:41:26.359 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:41:26.397 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:41:26.750 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:41:26.778 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:26.817 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:27.097 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:27.120 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:41:27.148 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:27.178 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:27.223 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:41:27.349 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:41:27.476 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:27.481 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:41:27.497 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:27.539 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:27.601 INFO:teuthology.orchestra.run.smithi129.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:27.783 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:41:27.811 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:41:27.850 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:41:27.854 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:41:27.854 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:41:28.337 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:41:28.365 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:41:28.387 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:41:28.403 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:41:28.479 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:41:28.748 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:41:28.775 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:41:28.806 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:41:28.814 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:28.950 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:41:29.438 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:41:29.465 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:41:29.505 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:41:30.170 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ppp. 2024-04-25T00:41:30.198 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:41:30.428 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:41:30.499 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:41:30.536 INFO:teuthology.orchestra.run.smithi129.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:41:30.775 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:41:30.792 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:41:30.824 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:41:31.027 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:41:31.040 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:41:31.052 INFO:teuthology.orchestra.run.smithi129.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:41:31.058 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:31.077 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:41:31.380 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:41:31.408 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:41:31.447 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:41:31.750 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:41:31.778 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:31.817 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:32.397 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:41:32.425 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:41:32.481 INFO:teuthology.orchestra.run.smithi196.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:41:32.510 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:41:32.979 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:41:33.007 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:41:33.049 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:34.464 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:41:34.481 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:41:34.523 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:41:34.767 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:41:34.795 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:41:34.833 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:41:35.077 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:41:35.105 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:41:35.143 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:41:35.449 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:41:35.462 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:41:35.496 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:41:35.757 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package tcl. 2024-04-25T00:41:35.785 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:41:35.828 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:41:36.176 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:41:36.204 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:41:36.243 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:41:36.605 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:41:36.622 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:41:36.664 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:41:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:41:37.070 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:41:37.116 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:41:37.388 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:41:37.598 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:41:37.701 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:41:37.800 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:37.926 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:41:38.068 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:41:38.194 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:38.703 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:41:38.783 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:41:39.146 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:41:39.147 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:41:39.611 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:41:39.746 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:41:39.888 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:41:40.014 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:41:40.140 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:41:40.241 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:41:40.342 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:41:40.451 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:41:41.106 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:41:41.224 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:41:41.405 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:41:41.459 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:41.568 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:41:41.694 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:41:41.828 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:41:41.954 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:41:42.070 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:41:42.139 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:41:42.265 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:41:42.441 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:41:42.551 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:41:42.660 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:41:42.811 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:41:42.937 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout:- enp3s0f1 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:41:43.402 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:43.521 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:41:43.833 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:41:44.152 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:41:44.623 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:41:44.775 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:41:44.892 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:41:44.993 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:41:45.820 INFO:teuthology.orchestra.run.smithi196.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:41:46.042 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:46.043 INFO:teuthology.orchestra.run.smithi196.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:41:46.238 INFO:teuthology.orchestra.run.smithi196.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:41:46.238 INFO:teuthology.orchestra.run.smithi196.stdout:Removing obsolete dictionary files: 2024-04-25T00:41:46.774 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:41:47.161 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:41:47.482 INFO:teuthology.orchestra.run.smithi196.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:41:47.510 INFO:teuthology.orchestra.run.smithi196.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout:This user must also own the server process. 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout:The default text search configuration will be set to "english". 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:47.565 INFO:teuthology.orchestra.run.smithi196.stdout:Data page checksums are disabled. 2024-04-25T00:41:47.566 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:47.566 INFO:teuthology.orchestra.run.smithi196.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:41:47.566 INFO:teuthology.orchestra.run.smithi196.stdout:creating subdirectories ... ok 2024-04-25T00:41:47.566 INFO:teuthology.orchestra.run.smithi196.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:41:47.582 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default max_connections ... 100 2024-04-25T00:41:47.601 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:41:47.612 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:41:47.612 INFO:teuthology.orchestra.run.smithi196.stdout:creating configuration files ... ok 2024-04-25T00:41:47.725 INFO:teuthology.orchestra.run.smithi196.stdout:running bootstrap script ... ok 2024-04-25T00:41:48.341 INFO:teuthology.orchestra.run.smithi196.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:41:49.132 INFO:teuthology.orchestra.run.smithi196.stdout:syncing data to disk ... ok 2024-04-25T00:41:49.515 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:41:52.276 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:52.402 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:41:52.528 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:41:52.637 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:41:52.738 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:41:52.967 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:41:52.967 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:41:53.499 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:41:53.583 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:41:53.910 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:41:54.063 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:41:55.440 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:41:55.548 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:41:56.044 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:41:56.049 DEBUG:teuthology.parallel:result is None 2024-04-25T00:41:56.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:41:56.194 DEBUG:teuthology.orchestra.run.smithi129:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:41:56.215 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:41:56.216 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:41:56.216 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:41:56.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:41:56.442 DEBUG:teuthology.orchestra.run.smithi196:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:41:56.461 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:41:56.461 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:41:56.461 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:41:56.463 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:41:56.463 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:56.463 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:41:56.478 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:56.478 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:41:56.518 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:41:56.518 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:56.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:41:56.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:41:56.589 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:56.590 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:41:56.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:41:56.656 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:41:56.656 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:56.656 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:41:56.669 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:41:56.724 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:56.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:41:56.738 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:41:56.793 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:41:56.793 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:56.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:41:56.806 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:41:56.860 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:56.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:41:56.873 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:41:56.930 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T00:41:57.016 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T00:41:57.016 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-04-25T00:41:57.018 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-04-25T00:41:57.031 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T00:41:57.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:41:57.075 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:41:57.088 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T00:41:57.088 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:41:57.131 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:41:57.151 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T00:41:57.151 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T00:41:57.151 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T00:41:57.151 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:57.152 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:41:57.190 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:41:57.264 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:57.264 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:41:57.271 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:41:57.349 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T00:41:57.349 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T00:41:57.350 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/workunit/dir-max-entries}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672276', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9cm981KG+zU/4NvE8B82NrdnmELXSd5LCeiKRt61nGN3j2j7n3e4/osGo0coIWjsx8Db8InqaJY4/MZpXbR18=', 'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAlKPAhN/nGw+u89+i8y+9N/255AGygtVzmIqfMLXr7G5zDal68Y4TRlSwWuJlU68iHc3UAPN7dtKSvmiZaXrU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'workunit': {'clients': {'client.0': ['fs/maxentries']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T00:41:57.350 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:41:57.353 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:41:57.393 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:41:57.399 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:41:57.449 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:57.449 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:41:57.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:41:57.458 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-04-25T00:41:57.505 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 00:35:19.550179067 +0000 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:35:19.294181353 +0000 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:35:19.294181353 +0000 2024-04-25T00:41:57.506 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2024-04-25T00:41:57.506 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:41:57.559 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T00:41:57.560 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T00:41:57.560 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000375782 s, 1.4 MB/s 2024-04-25T00:41:57.561 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:41:57.611 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 00:35:20.038174708 +0000 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:35:19.778177030 +0000 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:35:19.778177030 +0000 2024-04-25T00:41:57.658 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2024-04-25T00:41:57.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:41:57.712 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T00:41:57.712 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T00:41:57.712 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.00042633 s, 1.2 MB/s 2024-04-25T00:41:57.713 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:41:57.763 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 00:35:20.578169884 +0000 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:35:20.286172492 +0000 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:35:20.286172492 +0000 2024-04-25T00:41:57.814 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2024-04-25T00:41:57.815 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:41:57.868 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T00:41:57.868 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T00:41:57.868 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000447761 s, 1.1 MB/s 2024-04-25T00:41:57.869 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:41:57.920 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-04-25T00:41:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:41:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:57.967 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-04-25T00:41:57.967 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:57.968 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 00:35:21.082165383 +0000 2024-04-25T00:41:57.968 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:35:20.810167812 +0000 2024-04-25T00:41:57.968 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:35:20.810167812 +0000 2024-04-25T00:41:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2024-04-25T00:41:57.968 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T00:41:58.020 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000356972 s, 1.4 MB/s 2024-04-25T00:41:58.022 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:41:58.071 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:41:58.071 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:58.071 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:41:58.077 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:41:58.077 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 00:35:21.326819359 +0000 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 00:35:21.078821574 +0000 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 00:35:21.078821574 +0000 2024-04-25T00:41:58.126 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T00:41:58.126 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:41:58.179 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T00:41:58.180 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T00:41:58.180 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000416144 s, 1.2 MB/s 2024-04-25T00:41:58.181 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:41:58.230 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 00:35:21.862814571 +0000 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 00:35:21.598816929 +0000 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 00:35:21.598816929 +0000 2024-04-25T00:41:58.277 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T00:41:58.277 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:41:58.331 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T00:41:58.331 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T00:41:58.332 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000344198 s, 1.5 MB/s 2024-04-25T00:41:58.333 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:41:58.382 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 902 Links: 1 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 00:35:22.406809712 +0000 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 00:35:22.118812284 +0000 2024-04-25T00:41:58.429 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 00:35:22.118812284 +0000 2024-04-25T00:41:58.430 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T00:41:58.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T00:41:58.483 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000316454 s, 1.6 MB/s 2024-04-25T00:41:58.484 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:41:58.534 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 00:35:22.630807711 +0000 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 00:35:22.626807747 +0000 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 00:35:22.626807747 +0000 2024-04-25T00:41:58.581 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T00:41:58.582 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:41:58.635 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T00:41:58.635 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T00:41:58.635 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000384214 s, 1.3 MB/s 2024-04-25T00:41:58.636 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:41:58.686 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:41:58.686 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:41:58.686 INFO:tasks.ceph:Generating config... 2024-04-25T00:41:58.687 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T00:41:58.687 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T00:41:58.687 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T00:41:58.688 INFO:tasks.ceph:[client] pid file = 2024-04-25T00:41:58.688 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T00:41:58.688 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T00:41:58.688 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T00:41:58.689 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T00:41:58.689 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T00:41:58.689 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T00:41:58.689 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T00:41:58.689 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T00:41:58.689 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T00:41:58.690 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:41:58.690 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T00:41:58.690 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T00:41:58.690 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T00:41:58.690 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T00:41:58.690 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T00:41:58.690 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T00:41:58.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T00:41:58.725 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T00:41:58.739 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T00:41:58.816 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:41:58.874 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.129'), ('mon.b', '172.21.15.196'), ('mon.c', '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]')] 2024-04-25T00:41:58.876 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.129,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T00:41:58.877 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:58.878 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:41:58.922 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.129 --add b 172.21.15.196 --addv c '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: generated fsid aaa4363a-84af-4fd8-b222-fe97e0bf8f21 2024-04-25T00:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout:setting min_mon_release = quincy 2024-04-25T00:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 0 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:fsid aaa4363a-84af-4fd8-b222-fe97e0bf8f21 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:last_changed 2024-04-25T00:41:59.010287+0000 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-04-25T00:41:59.010287+0000 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:min_mon_release 17 (quincy) 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:election_strategy: 1 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:2: [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] mon.c 2024-04-25T00:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T00:41:59.009 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:41:59.057 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID aaa4363a-84af-4fd8-b222-fe97e0bf8f21... 2024-04-25T00:41:59.058 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:41:59.102 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-04-25T00:41:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = "172.21.15.129,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = aaa4363a-84af-4fd8-b222-fe97e0bf8f21 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-04-25T00:41:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 600 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug trim objects = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: bdev async discard = True 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: bdev enable discard = True 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore allocator = bitmap 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore block size = 96636764160 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore fsck on mount = True 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluefs = 1/20 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluestore = 1/20 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T00:41:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: debug rocksdb = 4/10 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd full ratio = 0.9 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = bluestore 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: osd op complaint time = 180 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-04-25T00:41:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: debug client = 20 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:41:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: mon down mkfs grace = 300 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: mon op complaint time = 120 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: client mount timeout = 600 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: debug client = 20 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = "" 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: rados mon op timeout = 900 2024-04-25T00:41:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: rados osd op timeout = 900 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.b] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.c] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout:[mds] 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: debug mds = 20 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: debug mds balancer = 20 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: mds debug frag = True 2024-04-25T00:41:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: mds debug scatterstat = True 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: mds op complaint time = 180 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: mds verify scatter = True 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: osd op complaint time = 180 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: rados mon op timeout = 900 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: rados osd op timeout = 900 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:41:59.142 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:41:59.143 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:41:59.144 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.129,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = aaa4363a-84af-4fd8-b222-fe97e0bf8f21 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.145 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.146 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: bdev async discard = True 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: bdev enable discard = True 2024-04-25T00:41:59.147 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore allocator = bitmap 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 1/20 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 1/20 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 4/10 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.148 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T00:41:59.149 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:41:59.150 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:41:59.151 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-25T00:41:59.152 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T00:41:59.153 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T00:41:59.154 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T00:41:59.155 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T00:41:59.250 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T00:41:59.250 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:59.250 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T00:41:59.256 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:59.256 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T00:41:59.302 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:41:59.302 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:59.302 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:41:59.302 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:41:59.367 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:41:59.367 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:41:59.414 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:41:59.414 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:59.414 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:41:59.414 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:41:59.437 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:41:59.437 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:41:59.485 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T00:41:59.486 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T00:41:59.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:41:59.532 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:41:59.577 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:41:59.601 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T00:41:59.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:41:59.644 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:41:59.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T00:41:59.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T00:41:59.787 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T00:41:59.790 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T00:41:59.845 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:41:59.891 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:41:59.894 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T00:41:59.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T00:42:00.031 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T00:42:00.033 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T00:42:00.087 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T00:42:00.087 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:42:00.121 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T00:42:00.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T00:42:00.203 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T00:42:00.213 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T00:42:00.251 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T00:42:00.262 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T00:42:00.336 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T00:42:00.348 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T00:42:00.348 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:42:00.348 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T00:42:00.361 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:00.361 INFO:tasks.ceph:role: osd.0 2024-04-25T00:42:00.361 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:42:00.361 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:00.425 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:00.875 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T00:42:00.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T00:42:00.877 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T00:42:00.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T00:42:00.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:00.981 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:00.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T00:42:01.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:01.035 INFO:tasks.ceph:role: osd.1 2024-04-25T00:42:01.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:42:01.035 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:01.099 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:01.559 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T00:42:01.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T00:42:01.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T00:42:01.701 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T00:42:01.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:01.713 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:01.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T00:42:01.769 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:01.769 INFO:tasks.ceph:role: osd.2 2024-04-25T00:42:01.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:42:01.769 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:42:01.836 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:01.836 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:01.837 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:02.322 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T00:42:02.324 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T00:42:02.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T00:42:02.484 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T00:42:02.495 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:02.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:02.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T00:42:02.549 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:02.549 INFO:tasks.ceph:role: osd.3 2024-04-25T00:42:02.549 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:42:02.549 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:02.612 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:03.020 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T00:42:03.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T00:42:03.023 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T00:42:03.125 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T00:42:03.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:03.134 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:03.135 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:03.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:03.243+0000 7f8c69c457c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T00:42:03.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:03.243+0000 7f8c69c457c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T00:42:03.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:03.243+0000 7f8c69c457c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:42:03.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:03.243+0000 7f8c69c457c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:42:03.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:03.243+0000 7f8c69c457c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:42:03.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:03.247+0000 7f8c69c457c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T00:42:06.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T00:42:06.219 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:06.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:06.299+0000 7fa7641737c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T00:42:06.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:06.299+0000 7fa7641737c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T00:42:06.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:06.299+0000 7fa7641737c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:42:06.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:06.299+0000 7fa7641737c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:42:06.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:06.299+0000 7fa7641737c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:42:06.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:06.299+0000 7fa7641737c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T00:42:09.276 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T00:42:09.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:09.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:09.411+0000 7f78120477c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T00:42:09.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:09.411+0000 7f78120477c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T00:42:09.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:09.411+0000 7f78120477c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:42:09.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:09.411+0000 7f78120477c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:42:09.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:09.411+0000 7f78120477c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:42:09.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:09.411+0000 7f78120477c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T00:42:12.401 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T00:42:12.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:12.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:12.542+0000 7f6abb1887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T00:42:12.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:12.542+0000 7f6abb1887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T00:42:12.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:12.542+0000 7f6abb1887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:42:12.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:12.542+0000 7f6abb1887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:42:12.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:12.542+0000 7f6abb1887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:42:12.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:12.542+0000 7f6abb1887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T00:42:15.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T00:42:15.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T00:42:15.589 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:15.590 INFO:tasks.ceph:role: osd.4 2024-04-25T00:42:15.590 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:42:15.590 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:42:15.656 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:15.656 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:15.657 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:16.118 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T00:42:16.120 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T00:42:16.120 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T00:42:16.225 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T00:42:16.234 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:16.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:16.235 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T00:42:16.288 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:16.288 INFO:tasks.ceph:role: osd.5 2024-04-25T00:42:16.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:42:16.288 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:42:16.348 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:16.349 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:16.818 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T00:42:16.821 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T00:42:16.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T00:42:16.956 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T00:42:16.967 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:16.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:16.968 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T00:42:17.021 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:17.021 INFO:tasks.ceph:role: osd.6 2024-04-25T00:42:17.021 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:42:17.021 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:17.085 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:17.086 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:17.086 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:17.558 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T00:42:17.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T00:42:17.560 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T00:42:17.699 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T00:42:17.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:17.711 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:17.711 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T00:42:17.765 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:42:17.765 INFO:tasks.ceph:role: osd.7 2024-04-25T00:42:17.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:42:17.765 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:42:17.829 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:42:17.830 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:42:17.830 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:42:18.245 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T00:42:18.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T00:42:18.248 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T00:42:18.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T00:42:18.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:42:18.356 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:42:18.356 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:18.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:18.463+0000 7fd69ff6f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T00:42:18.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:18.467+0000 7fd69ff6f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T00:42:18.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:18.467+0000 7fd69ff6f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:42:18.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:18.467+0000 7fd69ff6f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:42:18.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:18.467+0000 7fd69ff6f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:42:18.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:18.471+0000 7fd69ff6f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T00:42:21.459 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T00:42:21.473 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:21.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:21.555+0000 7f72e73307c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T00:42:21.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:21.555+0000 7f72e73307c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T00:42:21.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:21.555+0000 7f72e73307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:42:21.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:21.555+0000 7f72e73307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:42:21.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:21.555+0000 7f72e73307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:42:21.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:21.555+0000 7f72e73307c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T00:42:24.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T00:42:24.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:24.667 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:24.666+0000 7fb66fd2f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T00:42:24.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:24.666+0000 7fb66fd2f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T00:42:24.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:24.666+0000 7fb66fd2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:42:24.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:24.666+0000 7fb66fd2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:42:24.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:24.666+0000 7fb66fd2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:42:24.668 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:24.666+0000 7fb66fd2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T00:42:27.351 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:27.354 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:27.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T00:42:27.658 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:27.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:27.742+0000 7fad6f3957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T00:42:27.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:27.742+0000 7fad6f3957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T00:42:27.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:27.742+0000 7fad6f3957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:42:27.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:27.742+0000 7fad6f3957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:42:27.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:27.742+0000 7fad6f3957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:42:27.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:27.746+0000 7fad6f3957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T00:42:30.720 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T00:42:30.774 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T00:42:30.774 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:30.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T00:42:30.788 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:30.788 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T00:42:30.841 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:30.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T00:42:30.901 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:30.902 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T00:42:30.957 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:30.957 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T00:42:31.010 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:31.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T00:42:31.065 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:31.065 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T00:42:31.118 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.118 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T00:42:31.131 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.131 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T00:42:31.185 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T00:42:31.245 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.245 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T00:42:31.300 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T00:42:31.353 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.353 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T00:42:31.406 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.406 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T00:42:31.465 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:31.465 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T00:42:31.473 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:31.473 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T00:42:31.523 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.523 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T00:42:31.529 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:31.529 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T00:42:31.581 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T00:42:31.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:42:31.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.y] 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDXpilmXe7AHxAA1AVhKXYN2+89MObE0Xepag== 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a] 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDXpilmURedJhAAI38eiLLLi//rAw4v+eKwNg== 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c] 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDXpilm+EwaLxAAP9Dwh+taIfrAvMWPezzKpQ== 2024-04-25T00:42:31.629 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDbpilm5muVDhAAcqdosN+HdJ5ZbNntH+CwOQ== 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDepilmVMcGEhAAm0a8vGpbbfK+0cwg4nt+Uw== 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDhpilm+dimGBAAPDos60sN+GMHv6XNWPXPTg== 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDkpilm4wN4IBAAYZ8FnwnUAkJEZ27DbcS9Pw== 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDXpilmudKWIhAA2hkxsQredp2wsVFgAJ4nTA== 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b] 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDXpilmSv5SNRAA4nlwnQ4MYWzmL0G42qLtFQ== 2024-04-25T00:42:31.630 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDYpilminwLAhAAc/N5SzsL8EpUyU0OVm5nmQ== 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDqpilm81zLGxAAkbJ3q/CSKYCFBFs0Nt2nmg== 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDtpilmAbA5IRAAZVNjiXMyg/VI1iLb5CNc+Q== 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.6] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDwpilmoJXoJxAAWbhZcyW0YvmOHXgyKO41Xg== 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.7] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDzpilmiuR+LBAACP18YCkMjgkawdSE9GeuBw== 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDYpilmdDdwBxAAKO0LqjKOyMFBGvYHRvj3dQ== 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout:[client.2] 2024-04-25T00:42:31.631 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDYpilmzypSDBAAiVLWAj97GsGwfiNsC1rihA== 2024-04-25T00:42:31.632 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-04-25T00:42:31.632 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDYpilm/ZcqDxAAfK990n93joYUPl9ZPpRtqg== 2024-04-25T00:42:31.632 INFO:teuthology.orchestra.run.smithi129.stdout:[client.3] 2024-04-25T00:42:31.632 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDYpilme5dCFBAAttwxseD0pjPjPVspsS9weg== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.y] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDXpilmXe7AHxAA1AVhKXYN2+89MObE0Xepag== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDXpilmURedJhAAI38eiLLLi//rAw4v+eKwNg== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDXpilm+EwaLxAAP9Dwh+taIfrAvMWPezzKpQ== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.0] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDbpilm5muVDhAAcqdosN+HdJ5ZbNntH+CwOQ== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.1] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDepilmVMcGEhAAm0a8vGpbbfK+0cwg4nt+Uw== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.2] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDhpilm+dimGBAAPDos60sN+GMHv6XNWPXPTg== 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.3] 2024-04-25T00:42:31.637 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkpilm4wN4IBAAYZ8FnwnUAkJEZ27DbcS9Pw== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.x] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDXpilmudKWIhAA2hkxsQredp2wsVFgAJ4nTA== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDXpilmSv5SNRAA4nlwnQ4MYWzmL0G42qLtFQ== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDYpilminwLAhAAc/N5SzsL8EpUyU0OVm5nmQ== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.4] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDqpilm81zLGxAAkbJ3q/CSKYCFBFs0Nt2nmg== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.5] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDtpilmAbA5IRAAZVNjiXMyg/VI1iLb5CNc+Q== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.6] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDwpilmoJXoJxAAWbhZcyW0YvmOHXgyKO41Xg== 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.7] 2024-04-25T00:42:31.638 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDzpilmiuR+LBAACP18YCkMjgkawdSE9GeuBw== 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDYpilmdDdwBxAAKO0LqjKOyMFBGvYHRvj3dQ== 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout:[client.2] 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDYpilmzypSDBAAiVLWAj97GsGwfiNsC1rihA== 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDYpilm/ZcqDxAAfK990n93joYUPl9ZPpRtqg== 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout:[client.3] 2024-04-25T00:42:31.639 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDYpilme5dCFBAAttwxseD0pjPjPVspsS9weg== 2024-04-25T00:42:31.640 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:42:31.677 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:42:31.727 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:31.765 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:31.817 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:31.852 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:31.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:31.937 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:31.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.019 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:42:32.267 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:42:32.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:32.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:32.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:32.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:42:32.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.519 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.608 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.776 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:42:32.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:32.860 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:32.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:32.944 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:33.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:33.028 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:33.070 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:33.111 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:42:33.155 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T00:42:33.155 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T00:42:33.214 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:42:33.307 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T00:42:33.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T00:42:33.332 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:42:33.432 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T00:42:33.442 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T00:42:33.495 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:42:33.588 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T00:42:33.599 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:33.601 DEBUG:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:42:33.645 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T00:42:33.645 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T00:42:33.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T00:42:33.648 INFO:tasks.ceph.mon.a:Started 2024-04-25T00:42:33.648 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T00:42:33.648 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T00:42:33.691 INFO:tasks.ceph.mon.b:Started 2024-04-25T00:42:33.691 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T00:42:33.691 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T00:42:33.694 INFO:tasks.ceph.mon.c:Started 2024-04-25T00:42:33.694 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T00:42:33.694 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T00:42:33.694 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T00:42:33.696 INFO:tasks.ceph.mgr.y:Started 2024-04-25T00:42:33.697 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T00:42:33.697 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T00:42:33.699 INFO:tasks.ceph.mgr.x:Started 2024-04-25T00:42:33.699 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T00:42:33.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T00:42:33.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.809+0000 7f360e889640 1 Processor -- start 2024-04-25T00:42:33.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.809+0000 7f360e889640 1 -- start start 2024-04-25T00:42:33.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360e889640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360e889640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36080b5b60 con 0x7f360815c0f0 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360e889640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36080b5e40 con 0x7f360806ecf0 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360e889640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f36080b6120 con 0x7f36080cef70 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360ce26640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f3607fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f360815c0f0 0x7f36080b5310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:48518/0 (socket says 172.21.15.129:48518) 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f3607fff640 1 -- 172.21.15.129:0/1918982088 learned_addr learned my addr 172.21.15.129:0/1918982088 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 msgr2=0x7f36080cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 msgr2=0x7f36080cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:42:33.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:33.813+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:42:34.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.017+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.017+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 msgr2=0x7f36080cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T00:42:34.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.017+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 msgr2=0x7f36080cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:42:34.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.017+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:42:34.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.017+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:42:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758156516 0 0) 0x7f36080b5b60 con 0x7f360815c0f0 2024-04-25T00:42:34.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35f4003530 con 0x7f360815c0f0 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2613987129 0 0) 0x7f35f4003530 con 0x7f360815c0f0 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 msgr2=0x7f36080cf400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 --2- 172.21.15.129:0/1918982088 >> v2:172.21.15.129:3300/0 conn(0x7f36080cef70 0x7f36080cf400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 >> v1:172.21.15.196:6789/0 conn(0x7f360806ecf0 legacy=0x7f360815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36080b6400 con 0x7f360815c0f0 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f35f8001680 con 0x7f360815c0f0 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 >> v1:172.21.15.129:6789/0 conn(0x7f360815c0f0 legacy=0x7f36080b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f4003a10 0x7f35f4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f4004810 0x7f35f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35f40052f0 0x7f35f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36080b6120 con 0x7f35f40052f0 2024-04-25T00:42:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400c210 con 0x7f35f4004810 2024-04-25T00:42:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f35f400c410 con 0x7f35f4003a10 2024-04-25T00:42:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35f40052f0 0x7f35f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f3607fff640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f4003a10 0x7f35f4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f35ff7fe640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f4004810 0x7f35f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f4003a10 msgr2=0x7f35f4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f4003a10 0x7f35f4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f4004810 msgr2=0x7f35f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f4004810 0x7f35f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f4003530 con 0x7f35f40052f0 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f35ff7fe640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f4004810 0x7f35f4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:34.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.049+0000 7f3607fff640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f4003a10 0x7f35f4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:34.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360ce26640 1 -- 172.21.15.129:0/1918982088 reap_dead start 2024-04-25T00:42:34.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360ce26640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35f40052f0 0x7f35f400b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3600009a40 tx=0x7f360001cff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffc95f65fc1d566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8007c30 con 0x7f35f40052f0 2024-04-25T00:42:34.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36000ad440 con 0x7f35f40052f0 2024-04-25T00:42:34.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f36077fe640 1 -- 172.21.15.129:0/1918982088 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f00013a0 con 0x7f35f40052f0 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 -- 172.21.15.129:0/1918982088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35f40052f0 msgr2=0x7f35f400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35f40052f0 0x7f35f400b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3600009a40 tx=0x7f360001cff0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 -- 172.21.15.129:0/1918982088 shutdown_connections 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 --2- 172.21.15.129:0/1918982088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35f40052f0 0x7f35f400b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 -- 172.21.15.129:0/1918982088 >> 172.21.15.129:0/1918982088 conn(0x7f360806b210 msgr2=0x7f360806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 -- 172.21.15.129:0/1918982088 shutdown_connections 2024-04-25T00:42:34.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.057+0000 7f360e889640 1 -- 172.21.15.129:0/1918982088 wait complete. 2024-04-25T00:42:34.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 Processor -- start 2024-04-25T00:42:34.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 -- start start 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f40052f0 0x7f3608149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36080c7440 0x7f36080c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3608157290 0x7f3608157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36080cf790 con 0x7f3608157290 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36080cf610 con 0x7f36080c7440 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.061+0000 7f360e889640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f360814a180 con 0x7f35f40052f0 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f360ce26640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3608157290 0x7f3608157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f360ce26640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3608157290 0x7f3608157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59150/0 (socket says 172.21.15.129:59150) 2024-04-25T00:42:34.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f360ce26640 1 -- 172.21.15.129:0/2876605530 learned_addr learned my addr 172.21.15.129:0/2876605530 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:34.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36080c7440 0x7f36080c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f3607fff640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f40052f0 0x7f3608149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 -- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f40052f0 msgr2=0x7f3608149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f40052f0 0x7f3608149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 -- 172.21.15.129:0/2876605530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3608157290 msgr2=0x7f3608157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3608157290 0x7f3608157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 -- 172.21.15.129:0/2876605530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3608158000 con 0x7f36080c7440 2024-04-25T00:42:34.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.065+0000 7f35ff7fe640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36080c7440 0x7f36080c78c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35f000f5c0 tx=0x7f35f000f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f001a070 con 0x7f36080c7440 2024-04-25T00:42:34.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f00053e0 con 0x7f36080c7440 2024-04-25T00:42:34.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36081582e0 con 0x7f36080c7440 2024-04-25T00:42:34.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0007270 con 0x7f36080c7440 2024-04-25T00:42:34.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36081588a0 con 0x7f36080c7440 2024-04-25T00:42:34.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f35f001d070 con 0x7f36080c7440 2024-04-25T00:42:34.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35f0003a50 con 0x7f36080c7440 2024-04-25T00:42:34.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.069+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3608062e50 con 0x7f36080c7440 2024-04-25T00:42:34.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.077+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f35f0024020 con 0x7f36080c7440 2024-04-25T00:42:34.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.141+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3608075320 con 0x7f36080c7440 2024-04-25T00:42:34.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.149+0000 7f36057fa640 1 -- 172.21.15.129:0/2876605530 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f35f0003c50 con 0x7f36080c7440 2024-04-25T00:42:34.153 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2024-04-25T00:42:34.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36080c7440 msgr2=0x7f36080c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36080c7440 0x7f36080c78c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35f000f5c0 tx=0x7f35f000f9c0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 shutdown_connections 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3608157290 0x7f3608157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36080c7440 0x7f36080c78c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 --2- 172.21.15.129:0/2876605530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f40052f0 0x7f3608149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 >> 172.21.15.129:0/2876605530 conn(0x7f360806b210 msgr2=0x7f36080b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 shutdown_connections 2024-04-25T00:42:34.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:34.153+0000 7f360e889640 1 -- 172.21.15.129:0/2876605530 wait complete. 2024-04-25T00:42:34.172 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T00:42:34.172 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T00:42:34.173 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:34.173 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T00:42:34.183 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:34.183 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T00:42:34.235 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:34.235 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T00:42:34.260 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:34.258+0000 7f6e11787180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:42:34.275 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:34.273+0000 7fac369bd180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:42:34.285 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:42:34.285 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T00:42:34.335 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:34.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T00:42:34.349 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:34.349 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T00:42:34.406 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:34.406 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T00:42:34.461 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:42:34.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T00:42:34.517 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 102816b8-6061-4739-bbba-d256a4352e02 0 2024-04-25T00:42:34.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfae600640 1 Processor -- start 2024-04-25T00:42:34.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfae600640 1 -- start start 2024-04-25T00:42:34.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfae600640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdfa8160380 0x7fdfa8160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfae600640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa80cd360 con 0x7fdfa81531b0 2024-04-25T00:42:34.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfae600640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa80cd640 con 0x7fdfa80cce10 2024-04-25T00:42:34.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfae600640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa80cdc00 con 0x7fdfa8160380 2024-04-25T00:42:34.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa77fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdfa80cce10 0x7fdfa815c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44880/0 (socket says 172.21.15.196:44880) 2024-04-25T00:42:34.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa77fe640 1 -- 172.21.15.196:0/922586204 learned_addr learned my addr 172.21.15.196:0/922586204 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T00:42:34.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfacb9d640 1 --2- 172.21.15.196:0/922586204 >> v2:172.21.15.129:3300/0 conn(0x7fdfa8160380 0x7fdfa8160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182340444 0 0) 0x7fdfa80cd640 con 0x7fdfa80cce10 2024-04-25T00:42:34.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf90003530 con 0x7fdfa80cce10 2024-04-25T00:42:34.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1485058764 0 0) 0x7fdf90003530 con 0x7fdfa80cce10 2024-04-25T00:42:34.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 >> v2:172.21.15.129:3300/0 conn(0x7fdfa8160380 msgr2=0x7fdfa8160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 --2- 172.21.15.196:0/922586204 >> v2:172.21.15.129:3300/0 conn(0x7fdfa8160380 0x7fdfa8160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 >> v1:172.21.15.129:6789/0 conn(0x7fdfa81531b0 legacy=0x7fdfa80b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa80cd920 con 0x7fdfa80cce10 2024-04-25T00:42:34.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fdf9c001680 con 0x7fdfa80cce10 2024-04-25T00:42:34.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 >> v1:172.21.15.196:6789/0 conn(0x7fdfa80cce10 legacy=0x7fdfa815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf900039c0 0x7fdf90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf90004810 0x7fdf90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf900052f0 0x7fdf9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80cdc00 con 0x7fdf90004810 2024-04-25T00:42:34.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9000c1c0 con 0x7fdf900039c0 2024-04-25T00:42:34.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9000c3c0 con 0x7fdf900052f0 2024-04-25T00:42:34.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa7fff640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf900039c0 0x7fdf90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfacb9d640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf900052f0 0x7fdf9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.682+0000 7fdfa77fe640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf90004810 0x7fdf90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa7fff640 1 -- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf900052f0 msgr2=0x7fdf9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa7fff640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf900052f0 0x7fdf9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa7fff640 1 -- 172.21.15.196:0/922586204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf90004810 msgr2=0x7fdf90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa7fff640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf90004810 0x7fdf90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa7fff640 1 -- 172.21.15.196:0/922586204 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf90003530 con 0x7fdf900039c0 2024-04-25T00:42:34.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfacb9d640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf900052f0 0x7fdf9000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:34.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa77fe640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf90004810 0x7fdf90004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:34.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfacb9d640 1 -- 172.21.15.196:0/922586204 reap_dead start 2024-04-25T00:42:34.704 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa7fff640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf900039c0 0x7fdf90003de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0001e40 tx=0x7fdfa001c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5dfabc567902d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00b0070 con 0x7fdf900039c0 2024-04-25T00:42:34.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa0003b80 con 0x7fdf900039c0 2024-04-25T00:42:34.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfa6ffd640 1 -- 172.21.15.196:0/922586204 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0001a90 con 0x7fdf900039c0 2024-04-25T00:42:34.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 -- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf900039c0 msgr2=0x7fdf90003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf900039c0 0x7fdf90003de0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0001e40 tx=0x7fdfa001c720 comp rx=0 tx=0).stop 2024-04-25T00:42:34.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 -- 172.21.15.196:0/922586204 shutdown_connections 2024-04-25T00:42:34.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 --2- 172.21.15.196:0/922586204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf900039c0 0x7fdf90003de0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 -- 172.21.15.196:0/922586204 >> 172.21.15.196:0/922586204 conn(0x7fdfa8070690 msgr2=0x7fdfa8070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 -- 172.21.15.196:0/922586204 shutdown_connections 2024-04-25T00:42:34.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.686+0000 7fdfae600640 1 -- 172.21.15.196:0/922586204 wait complete. 2024-04-25T00:42:34.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 Processor -- start 2024-04-25T00:42:34.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- start start 2024-04-25T00:42:34.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf900039c0 0x7fdfa8154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 0x7fdfa813c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa81565d0 0x7fdfa815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.708 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80b4a70 con 0x7fdf900039c0 2024-04-25T00:42:34.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa806f360 con 0x7fdfa81565d0 2024-04-25T00:42:34.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa80b4bf0 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 0x7fdfa813c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfacb9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa81565d0 0x7fdfa815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 0x7fdfa813c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41200/0 (socket says 172.21.15.196:41200) 2024-04-25T00:42:34.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 -- 172.21.15.196:0/3977113120 learned_addr learned my addr 172.21.15.196:0/3977113120 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:34.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa7fff640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf900039c0 0x7fdfa8154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 -- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa81565d0 msgr2=0x7fdfa815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa81565d0 0x7fdfa815e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 -- 172.21.15.196:0/3977113120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf900039c0 msgr2=0x7fdfa8154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf900039c0 0x7fdfa8154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 -- 172.21.15.196:0/3977113120 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa8155530 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa77fe640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 0x7fdfa813c040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00e780 tx=0x7fdf9c00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00ee30 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9c018070 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa8155810 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.712 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00cab0 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa8155dd0 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf9c00cc50 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fdf9c0135e0 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.690+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa8062e50 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.714 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.698+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdf9c0138f0 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.738 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:34.737+0000 7fac369bd180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:42:34.752 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:34.750+0000 7f6e11787180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:42:34.768 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.766+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "102816b8-6061-4739-bbba-d256a4352e02", "id": 0} v 0) -- 0x7fdfa8075320 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.775 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfa4ff9640 1 -- 172.21.15.196:0/3977113120 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "102816b8-6061-4739-bbba-d256a4352e02", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fdf9c013b30 con 0x7fdfa81e5ac0 2024-04-25T00:42:34.776 INFO:teuthology.orchestra.run.smithi196.stdout:0 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 msgr2=0x7fdfa813c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 0x7fdfa813c040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00e780 tx=0x7fdf9c00ec40 comp rx=0 tx=0).stop 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 shutdown_connections 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfa81e5ac0 0x7fdfa813c040 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa81565d0 0x7fdfa815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 --2- 172.21.15.196:0/3977113120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf900039c0 0x7fdfa8154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 >> 172.21.15.196:0/3977113120 conn(0x7fdfa8070690 msgr2=0x7fdfa80cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 shutdown_connections 2024-04-25T00:42:34.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.774+0000 7fdfae600640 1 -- 172.21.15.196:0/3977113120 wait complete. 2024-04-25T00:42:34.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 633ce22e-1391-43b9-af52-cae9377ac030 1 2024-04-25T00:42:34.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 Processor -- start 2024-04-25T00:42:34.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 -- start start 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 0x7f2a5015f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 0x7f2a50072460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5006f240 con 0x7f2a50070690 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5006d980 con 0x7f2a50073070 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a56287640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5006f520 con 0x7f2a50071ff0 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a54824640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 0x7f2a50072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a4ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 0x7f2a5015f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a4ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 0x7f2a5015f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54144/0 (socket says 172.21.15.196:54144) 2024-04-25T00:42:34.909 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a54824640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 0x7f2a50072460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41210/0 (socket says 172.21.15.196:41210) 2024-04-25T00:42:34.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a4ffff640 1 -- 172.21.15.196:0/606874037 learned_addr learned my addr 172.21.15.196:0/606874037 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:34.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a54824640 1 -- 172.21.15.196:0/606874037 >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 msgr2=0x7f2a5015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a54824640 1 --2- 172.21.15.196:0/606874037 >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 0x7f2a5015f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a54824640 1 -- 172.21.15.196:0/606874037 >> v1:172.21.15.129:6789/0 conn(0x7f2a50070690 legacy=0x7f2a5006de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:34.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a54824640 1 -- 172.21.15.196:0/606874037 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5006d610 con 0x7f2a50071ff0 2024-04-25T00:42:34.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.906+0000 7f2a4ffff640 1 --2- 172.21.15.196:0/606874037 >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 0x7f2a5015f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:34.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a54824640 1 --2- 172.21.15.196:0/606874037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 0x7f2a50072460 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009d90 tx=0x7f2a4801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc7b9656f9e4de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a4effd640 1 -- 172.21.15.196:0/606874037 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a480b2070 con 0x7f2a50071ff0 2024-04-25T00:42:34.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a4effd640 1 -- 172.21.15.196:0/606874037 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a480a5830 con 0x7f2a50071ff0 2024-04-25T00:42:34.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a4effd640 1 -- 172.21.15.196:0/606874037 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a480a5c00 con 0x7f2a50071ff0 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 -- 172.21.15.196:0/606874037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 msgr2=0x7f2a50072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 --2- 172.21.15.196:0/606874037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 0x7f2a50072460 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009d90 tx=0x7f2a4801c970 comp rx=0 tx=0).stop 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 -- 172.21.15.196:0/606874037 shutdown_connections 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 --2- 172.21.15.196:0/606874037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50071ff0 0x7f2a50072460 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 --2- 172.21.15.196:0/606874037 >> v2:172.21.15.196:3300/0 conn(0x7f2a50073070 0x7f2a5015f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 -- 172.21.15.196:0/606874037 >> 172.21.15.196:0/606874037 conn(0x7f2a500b2ed0 msgr2=0x7f2a500b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 -- 172.21.15.196:0/606874037 shutdown_connections 2024-04-25T00:42:34.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 -- 172.21.15.196:0/606874037 wait complete. 2024-04-25T00:42:34.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 Processor -- start 2024-04-25T00:42:34.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.910+0000 7f2a56287640 1 -- start start 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50070690 0x7f2a50155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a50071ff0 0x7f2a50154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a50073070 0x7f2a5015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5016b420 con 0x7f2a50073070 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5016b5a0 con 0x7f2a50071ff0 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5016b720 con 0x7f2a50070690 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50070690 0x7f2a50155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a50071ff0 0x7f2a50154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50070690 0x7f2a50155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41224/0 (socket says 172.21.15.196:41224) 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4ffff640 1 -- 172.21.15.196:0/3084465832 learned_addr learned my addr 172.21.15.196:0/3084465832 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:34.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a54824640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a50073070 0x7f2a5015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 -- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50070690 msgr2=0x7f2a50155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50070690 0x7f2a50155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 -- 172.21.15.196:0/3084465832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a50073070 msgr2=0x7f2a5015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a50073070 0x7f2a5015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 -- 172.21.15.196:0/3084465832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a501548d0 con 0x7f2a50071ff0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a54824640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a50073070 0x7f2a5015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4f7fe640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a50071ff0 0x7f2a50154060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c3c0 tx=0x7f2a4400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440052d0 con 0x7f2a50071ff0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a44005470 con 0x7f2a50071ff0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a50154bb0 con 0x7f2a50071ff0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44007260 con 0x7f2a50071ff0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5015ad70 con 0x7f2a50071ff0 2024-04-25T00:42:34.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a44007400 con 0x7f2a50071ff0 2024-04-25T00:42:34.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f2a44003a90 con 0x7f2a50071ff0 2024-04-25T00:42:34.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.914+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a50062e50 con 0x7f2a50071ff0 2024-04-25T00:42:34.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.918+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a44003ce0 con 0x7f2a50071ff0 2024-04-25T00:42:34.926 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:34.921+0000 7fac369bd180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:42:34.961 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:34.958+0000 7f6e11787180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:42:34.985 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.986+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "633ce22e-1391-43b9-af52-cae9377ac030", "id": 1} v 0) -- 0x7f2a500b4de0 con 0x7f2a50071ff0 2024-04-25T00:42:34.992 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a4cff9640 1 -- 172.21.15.196:0/3084465832 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "633ce22e-1391-43b9-af52-cae9377ac030", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f2a440067e0 con 0x7f2a50071ff0 2024-04-25T00:42:34.992 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-25T00:42:34.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a50071ff0 msgr2=0x7f2a50154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:34.993 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a50071ff0 0x7f2a50154060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c3c0 tx=0x7f2a4400c880 comp rx=0 tx=0).stop 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 shutdown_connections 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a50073070 0x7f2a5015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a50071ff0 0x7f2a50154060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 --2- 172.21.15.196:0/3084465832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a50070690 0x7f2a50155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.990+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 >> 172.21.15.196:0/3084465832 conn(0x7f2a500b2ed0 msgr2=0x7f2a500b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.994+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 shutdown_connections 2024-04-25T00:42:34.994 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:34.994+0000 7f2a56287640 1 -- 172.21.15.196:0/3084465832 wait complete. 2024-04-25T00:42:35.010 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 305e1314-691a-443f-b9c1-f78c3fd27f40 2 2024-04-25T00:42:35.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9ca8dc640 1 Processor -- start 2024-04-25T00:42:35.123 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9ca8dc640 1 -- start start 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9ca8dc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff9c415c0f0 0x7ff9c40b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9ca8dc640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9c40b5b60 con 0x7ff9c406ecf0 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9ca8dc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9c40b5e40 con 0x7ff9c40cef70 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9ca8dc640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff9c40b6120 con 0x7ff9c415c0f0 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c8e79640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff9c40cef70 0x7ff9c40b6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44894/0 (socket says 172.21.15.196:44894) 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c8e79640 1 -- 172.21.15.196:0/4287354227 learned_addr learned my addr 172.21.15.196:0/4287354227 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c3fff640 1 --2- 172.21.15.196:0/4287354227 >> v2:172.21.15.129:3300/0 conn(0x7ff9c415c0f0 0x7ff9c40b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562116666 0 0) 0x7ff9c40b5e40 con 0x7ff9c40cef70 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9b0003530 con 0x7ff9c40cef70 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 342830566 0 0) 0x7ff9b0003530 con 0x7ff9c40cef70 2024-04-25T00:42:35.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 >> v2:172.21.15.129:3300/0 conn(0x7ff9c415c0f0 msgr2=0x7ff9c40b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 --2- 172.21.15.196:0/4287354227 >> v2:172.21.15.129:3300/0 conn(0x7ff9c415c0f0 0x7ff9c40b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 >> v1:172.21.15.129:6789/0 conn(0x7ff9c406ecf0 legacy=0x7ff9c415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c40b6400 con 0x7ff9c40cef70 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7ff9bc001a00 con 0x7ff9c40cef70 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 >> v1:172.21.15.196:6789/0 conn(0x7ff9c40cef70 legacy=0x7ff9c40b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0003a10 0x7ff9b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b0004810 0x7ff9b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0005220 0x7ff9b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40b6120 con 0x7ff9b0003a10 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b000c170 con 0x7ff9b0004810 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b000c370 con 0x7ff9b0005220 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c37fe640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b0004810 0x7ff9b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.122+0000 7ff9c8e79640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0005220 0x7ff9b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c37fe640 1 -- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0005220 msgr2=0x7ff9b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c37fe640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0005220 0x7ff9b000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c37fe640 1 -- 172.21.15.196:0/4287354227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0003a10 msgr2=0x7ff9b0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c37fe640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0003a10 0x7ff9b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c37fe640 1 -- 172.21.15.196:0/4287354227 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b0003530 con 0x7ff9b0004810 2024-04-25T00:42:35.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c8e79640 1 -- 172.21.15.196:0/4287354227 reap_dead start 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c37fe640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b0004810 0x7ff9b0004c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac001750 tx=0x7ff9ac01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb390b0baf27d73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0013a0 con 0x7ff9b0004810 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ac0052f0 con 0x7ff9b0004810 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9c2ffd640 1 -- 172.21.15.196:0/4287354227 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac003840 con 0x7ff9b0004810 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b0004810 msgr2=0x7ff9b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b0004810 0x7ff9b0004c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac001750 tx=0x7ff9ac01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/4287354227 shutdown_connections 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 --2- 172.21.15.196:0/4287354227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b0004810 0x7ff9b0004c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/4287354227 >> 172.21.15.196:0/4287354227 conn(0x7ff9c406b210 msgr2=0x7ff9c406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/4287354227 shutdown_connections 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.126+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/4287354227 wait complete. 2024-04-25T00:42:35.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 Processor -- start 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- start start 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 0x7ff9c41f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9c414e320 0x7ff9c415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c415dc80 0x7ff9c41f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40b6f70 con 0x7ff9c414e320 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40b6df0 con 0x7ff9c415dc80 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c40b70f0 con 0x7ff9b0004810 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 0x7ff9c41f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 0x7ff9c41f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41248/0 (socket says 172.21.15.196:41248) 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 -- 172.21.15.196:0/1599690822 learned_addr learned my addr 172.21.15.196:0/1599690822 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c8e79640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c415dc80 0x7ff9c41f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c37fe640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9c414e320 0x7ff9c415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.134 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 -- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c415dc80 msgr2=0x7ff9c41f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c415dc80 0x7ff9c41f55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 -- 172.21.15.196:0/1599690822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9c414e320 msgr2=0x7ff9c415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9c414e320 0x7ff9c415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 -- 172.21.15.196:0/1599690822 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41f6240 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c8e79640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c415dc80 0x7ff9c41f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c3fff640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 0x7ff9c41f9730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e770 tx=0x7ff9b400ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4017070 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b400ee80 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41f6520 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400cae0 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41f9210 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9b401a070 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7ff9b401c870 con 0x7ff9b0004810 2024-04-25T00:42:35.135 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.130+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c4062e50 con 0x7ff9b0004810 2024-04-25T00:42:35.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.134+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff9b400cd20 con 0x7ff9b0004810 2024-04-25T00:42:35.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.218+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "305e1314-691a-443f-b9c1-f78c3fd27f40", "id": 2} v 0) -- 0x7ff9c4075320 con 0x7ff9b0004810 2024-04-25T00:42:35.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9c0ff9640 1 -- 172.21.15.196:0/1599690822 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "305e1314-691a-443f-b9c1-f78c3fd27f40", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7ff9b401c520 con 0x7ff9b0004810 2024-04-25T00:42:35.227 INFO:teuthology.orchestra.run.smithi196.stdout:2 2024-04-25T00:42:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 msgr2=0x7ff9c41f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 0x7ff9c41f9730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e770 tx=0x7ff9b400ec30 comp rx=0 tx=0).stop 2024-04-25T00:42:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 shutdown_connections 2024-04-25T00:42:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9c415dc80 0x7ff9c41f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9c414e320 0x7ff9c415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 --2- 172.21.15.196:0/1599690822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b0004810 0x7ff9c41f9730 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 >> 172.21.15.196:0/1599690822 conn(0x7ff9c406b210 msgr2=0x7ff9c406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 shutdown_connections 2024-04-25T00:42:35.229 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.226+0000 7ff9ca8dc640 1 -- 172.21.15.196:0/1599690822 wait complete. 2024-04-25T00:42:35.243 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new fdb3a2ff-3539-488c-b764-c8ad0ea448c0 3 2024-04-25T00:42:35.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4c322640 1 Processor -- start 2024-04-25T00:42:35.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4c322640 1 -- start start 2024-04-25T00:42:35.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4c322640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fef440b4870 0x7fef440cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4c322640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef440ae870 con 0x7fef440adfa0 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4c322640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef440aeb10 con 0x7fef4415e1d0 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4c322640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fef440b4d70 con 0x7fef440b4870 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fef4415e1d0 0x7fef440ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44900/0 (socket says 172.21.15.196:44900) 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 -- 172.21.15.196:0/3322982740 learned_addr learned my addr 172.21.15.196:0/3322982740 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a8bf640 1 --2- 172.21.15.196:0/3322982740 >> v2:172.21.15.129:3300/0 conn(0x7fef440b4870 0x7fef440cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203408662 0 0) 0x7fef440aeb10 con 0x7fef4415e1d0 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef30003530 con 0x7fef4415e1d0 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3897378197 0 0) 0x7fef30003530 con 0x7fef4415e1d0 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 >> v2:172.21.15.129:3300/0 conn(0x7fef440b4870 msgr2=0x7fef440cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 --2- 172.21.15.196:0/3322982740 >> v2:172.21.15.129:3300/0 conn(0x7fef440b4870 0x7fef440cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 >> v1:172.21.15.129:6789/0 conn(0x7fef440adfa0 legacy=0x7fef440cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440aedf0 con 0x7fef4415e1d0 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fef40001680 con 0x7fef4415e1d0 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 >> v1:172.21.15.196:6789/0 conn(0x7fef4415e1d0 legacy=0x7fef440ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef30004810 0x7fef30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef300051d0 0x7fef3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef440b4d70 con 0x7fef30004810 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef3000c070 con 0x7fef300039c0 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef3000c270 con 0x7fef300051d0 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a8bf640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef300051d0 0x7fef3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef498bd640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef30004810 0x7fef30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 -- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef300051d0 msgr2=0x7fef3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef300051d0 0x7fef3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 -- 172.21.15.196:0/3322982740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef30004810 msgr2=0x7fef30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef30004810 0x7fef30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a0be640 1 -- 172.21.15.196:0/3322982740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef30003530 con 0x7fef300039c0 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a8bf640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef300051d0 0x7fef3000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:35.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.354+0000 7fef4a8bf640 1 -- 172.21.15.196:0/3322982740 reap_dead start 2024-04-25T00:42:35.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4a0be640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef30003de0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c1c0 tx=0x7fef4001f390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f24db933d050945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40007d00 con 0x7fef300039c0 2024-04-25T00:42:35.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef40005ae0 con 0x7fef300039c0 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef490bc640 1 -- 172.21.15.196:0/3322982740 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40007070 con 0x7fef300039c0 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 -- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 msgr2=0x7fef30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef30003de0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c1c0 tx=0x7fef4001f390 comp rx=0 tx=0).stop 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 -- 172.21.15.196:0/3322982740 shutdown_connections 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 --2- 172.21.15.196:0/3322982740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef30003de0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 -- 172.21.15.196:0/3322982740 >> 172.21.15.196:0/3322982740 conn(0x7fef44070690 msgr2=0x7fef44070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 -- 172.21.15.196:0/3322982740 shutdown_connections 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 -- 172.21.15.196:0/3322982740 wait complete. 2024-04-25T00:42:35.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 Processor -- start 2024-04-25T00:42:35.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.358+0000 7fef4c322640 1 -- start start 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef441580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 0x7fef441470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef441f7a40 0x7fef441f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef4415e730 con 0x7fef441f7a40 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef4415e5b0 con 0x7fef300039c0 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef44073680 con 0x7fef44146c40 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 0x7fef441470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.364 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 0x7fef441470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41274/0 (socket says 172.21.15.196:41274) 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 -- 172.21.15.196:0/1753526001 learned_addr learned my addr 172.21.15.196:0/1753526001 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4a0be640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef441580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 -- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 msgr2=0x7fef441580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef441580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4a8bf640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef441f7a40 0x7fef441f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 -- 172.21.15.196:0/1753526001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef441f7a40 msgr2=0x7fef441f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef441f7a40 0x7fef441f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 -- 172.21.15.196:0/1753526001 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef441536d0 con 0x7fef44146c40 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef498bd640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 0x7fef441470c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef34000c10 tx=0x7fef3400f980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3401a070 con 0x7fef44146c40 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef34005320 con 0x7fef44146c40 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34007270 con 0x7fef44146c40 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441586d0 con 0x7fef44146c40 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fef44156a60 con 0x7fef44146c40 2024-04-25T00:42:35.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef3401d070 con 0x7fef44146c40 2024-04-25T00:42:35.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fef34003ab0 con 0x7fef44146c40 2024-04-25T00:42:35.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.362+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef44062e50 con 0x7fef44146c40 2024-04-25T00:42:35.366 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.366+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fef34024070 con 0x7fef44146c40 2024-04-25T00:42:35.433 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.434+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fdb3a2ff-3539-488c-b764-c8ad0ea448c0", "id": 3} v 0) -- 0x7fef44075320 con 0x7fef44146c40 2024-04-25T00:42:35.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef3affd640 1 -- 172.21.15.196:0/1753526001 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fdb3a2ff-3539-488c-b764-c8ad0ea448c0", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fef34007410 con 0x7fef44146c40 2024-04-25T00:42:35.439 INFO:teuthology.orchestra.run.smithi196.stdout:3 2024-04-25T00:42:35.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 msgr2=0x7fef441470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 0x7fef441470c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef34000c10 tx=0x7fef3400f980 comp rx=0 tx=0).stop 2024-04-25T00:42:35.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 shutdown_connections 2024-04-25T00:42:35.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef44146c40 0x7fef441470c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef441f7a40 0x7fef441f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 --2- 172.21.15.196:0/1753526001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef300039c0 0x7fef441580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 >> 172.21.15.196:0/1753526001 conn(0x7fef44070690 msgr2=0x7fef4406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 shutdown_connections 2024-04-25T00:42:35.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.438+0000 7fef4c322640 1 -- 172.21.15.196:0/1753526001 wait complete. 2024-04-25T00:42:35.455 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 514c45e0-59e1-432e-bf2e-c2df9ce352cb 4 2024-04-25T00:42:35.466 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.465+0000 7fac369bd180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:42:35.520 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.517+0000 7fac369bd180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:42:35.560 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.558+0000 7f6e11787180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:42:35.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c9de31640 1 Processor -- start 2024-04-25T00:42:35.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c9de31640 1 -- start start 2024-04-25T00:42:35.572 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.569+0000 7fac369bd180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c9de31640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2c9815c0f0 0x7f2c980b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c9de31640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980b5b60 con 0x7f2c980cef70 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c9de31640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980b5e40 con 0x7f2c9806ecf0 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c9de31640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f2c980b6120 con 0x7f2c9815c0f0 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c96ffd640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f2c9806ecf0 0x7f2c9815c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:44914/0 (socket says 172.21.15.196:44914) 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c96ffd640 1 -- 172.21.15.196:0/1305299604 learned_addr learned my addr 172.21.15.196:0/1305299604 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1305299604 >> v2:172.21.15.129:3300/0 conn(0x7f2c9815c0f0 0x7f2c980b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081270345 0 0) 0x7f2c980b5e40 con 0x7f2c9806ecf0 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c80003530 con 0x7f2c9806ecf0 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 136442647 0 0) 0x7f2c80003530 con 0x7f2c9806ecf0 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 >> v2:172.21.15.129:3300/0 conn(0x7f2c9815c0f0 msgr2=0x7f2c980b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 --2- 172.21.15.196:0/1305299604 >> v2:172.21.15.129:3300/0 conn(0x7f2c9815c0f0 0x7f2c980b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 >> v1:172.21.15.129:6789/0 conn(0x7f2c980cef70 legacy=0x7f2c980b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.570+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c980b6400 con 0x7f2c9806ecf0 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f2c8c001680 con 0x7f2c9806ecf0 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 >> v1:172.21.15.196:6789/0 conn(0x7f2c9806ecf0 legacy=0x7f2c9815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c800039c0 0x7f2c80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c80004810 0x7f2c80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800051d0 0x7f2c8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980b6120 con 0x7f2c80004810 2024-04-25T00:42:35.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8000c170 con 0x7f2c800039c0 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8000c370 con 0x7f2c800051d0 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c800039c0 0x7f2c80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c97fff640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800051d0 0x7f2c8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c96ffd640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c80004810 0x7f2c80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 -- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800051d0 msgr2=0x7f2c8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800051d0 0x7f2c8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 -- 172.21.15.196:0/1305299604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c80004810 msgr2=0x7f2c80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c80004810 0x7f2c80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 -- 172.21.15.196:0/1305299604 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c80003530 con 0x7f2c800039c0 2024-04-25T00:42:35.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c97fff640 1 -- 172.21.15.196:0/1305299604 reap_dead start 2024-04-25T00:42:35.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c800039c0 0x7f2c80003de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ac0 tx=0x7f2c9001ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7b2c647f527e89d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900b2070 con 0x7f2c800039c0 2024-04-25T00:42:35.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c900a5550 con 0x7f2c800039c0 2024-04-25T00:42:35.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c967fc640 1 -- 172.21.15.196:0/1305299604 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90009690 con 0x7f2c800039c0 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 -- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c800039c0 msgr2=0x7f2c80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c800039c0 0x7f2c80003de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ac0 tx=0x7f2c9001ca70 comp rx=0 tx=0).stop 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 -- 172.21.15.196:0/1305299604 shutdown_connections 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 --2- 172.21.15.196:0/1305299604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c800039c0 0x7f2c80003de0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 -- 172.21.15.196:0/1305299604 >> 172.21.15.196:0/1305299604 conn(0x7f2c9806b210 msgr2=0x7f2c9806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 -- 172.21.15.196:0/1305299604 shutdown_connections 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.574+0000 7f2c9de31640 1 -- 172.21.15.196:0/1305299604 wait complete. 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 Processor -- start 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- start start 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 0x7f2c98072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c9814a900 0x7f2c981f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c9814a170 0x7f2c98155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980b6f70 con 0x7f2c9814a900 2024-04-25T00:42:35.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980b6df0 con 0x7f2c9814a170 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2c980b70f0 con 0x7f2c800039c0 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 0x7f2c98072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c9814a170 0x7f2c98155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 0x7f2c98072c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41294/0 (socket says 172.21.15.196:41294) 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 -- 172.21.15.196:0/1234058605 learned_addr learned my addr 172.21.15.196:0/1234058605 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c96ffd640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c9814a900 0x7f2c981f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 -- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c9814a170 msgr2=0x7f2c98155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c9814a170 0x7f2c98155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 -- 172.21.15.196:0/1234058605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c9814a900 msgr2=0x7f2c981f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c9814a900 0x7f2c981f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 -- 172.21.15.196:0/1234058605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c98071ba0 con 0x7f2c800039c0 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c977fe640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 0x7f2c98072c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ac0 tx=0x7f2c90019810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900b2040 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c900ad440 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c900ad830 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c98071e80 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c98072440 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c900c0060 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f2c900b8b00 con 0x7f2c800039c0 2024-04-25T00:42:35.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.578+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c98073180 con 0x7f2c800039c0 2024-04-25T00:42:35.584 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.582+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2c900d7dd0 con 0x7f2c800039c0 2024-04-25T00:42:35.618 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.618+0000 7f6e11787180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:42:35.650 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.645+0000 7fac369bd180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:42:35.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.650+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "514c45e0-59e1-432e-bf2e-c2df9ce352cb", "id": 4} v 0) -- 0x7f2c98062e50 con 0x7f2c800039c0 2024-04-25T00:42:35.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c7ffff640 1 -- 172.21.15.196:0/1234058605 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "514c45e0-59e1-432e-bf2e-c2df9ce352cb", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f2c900c60b0 con 0x7f2c800039c0 2024-04-25T00:42:35.658 INFO:teuthology.orchestra.run.smithi196.stdout:4 2024-04-25T00:42:35.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 msgr2=0x7f2c98072c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 0x7f2c98072c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ac0 tx=0x7f2c90019810 comp rx=0 tx=0).stop 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 shutdown_connections 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c9814a170 0x7f2c98155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c9814a900 0x7f2c981f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 --2- 172.21.15.196:0/1234058605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2c800039c0 0x7f2c98072c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 >> 172.21.15.196:0/1234058605 conn(0x7f2c9806b210 msgr2=0x7f2c9806c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 shutdown_connections 2024-04-25T00:42:35.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.658+0000 7f2c9de31640 1 -- 172.21.15.196:0/1234058605 wait complete. 2024-04-25T00:42:35.673 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.674+0000 7f6e11787180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:42:35.674 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 6b19aaa8-4475-4735-a1d5-f2debb8b5165 5 2024-04-25T00:42:35.702 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.701+0000 7fac369bd180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:42:35.753 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.750+0000 7f6e11787180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:42:35.779 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.777+0000 7fac369bd180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:42:35.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 Processor -- start 2024-04-25T00:42:35.788 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 -- start start 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff5740cffa0 0x7ff5740d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5740b01f0 0x7ff5740b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57415b410 con 0x7ff57406ba90 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff57415bd20 con 0x7ff5740cffa0 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff57a7df640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff5740d0c90 con 0x7ff5740b01f0 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff578d7c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5740b01f0 0x7ff5740b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff578d7c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5740b01f0 0x7ff5740b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54218/0 (socket says 172.21.15.196:54218) 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff578d7c640 1 -- 172.21.15.196:0/2751699077 learned_addr learned my addr 172.21.15.196:0/2751699077 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff56bfff640 1 --2- 172.21.15.196:0/2751699077 >> v2:172.21.15.129:3300/0 conn(0x7ff5740cffa0 0x7ff5740d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff578d7c640 1 -- 172.21.15.196:0/2751699077 >> v2:172.21.15.129:3300/0 conn(0x7ff5740cffa0 msgr2=0x7ff5740d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff578d7c640 1 --2- 172.21.15.196:0/2751699077 >> v2:172.21.15.129:3300/0 conn(0x7ff5740cffa0 0x7ff5740d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.786+0000 7ff578d7c640 1 -- 172.21.15.196:0/2751699077 >> v1:172.21.15.129:6789/0 conn(0x7ff57406ba90 legacy=0x7ff57406be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:35.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff578d7c640 1 -- 172.21.15.196:0/2751699077 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57415b6d0 con 0x7ff5740b01f0 2024-04-25T00:42:35.791 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff578d7c640 1 --2- 172.21.15.196:0/2751699077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff5740b0680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5740aff80 tx=0x7ff56c01c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d3454e298b4a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff5737fe640 1 -- 172.21.15.196:0/2751699077 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0a5780 con 0x7ff5740b01f0 2024-04-25T00:42:35.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff5737fe640 1 -- 172.21.15.196:0/2751699077 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56c0a5920 con 0x7ff5740b01f0 2024-04-25T00:42:35.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff5737fe640 1 -- 172.21.15.196:0/2751699077 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0a5cf0 con 0x7ff5740b01f0 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 -- 172.21.15.196:0/2751699077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 msgr2=0x7ff5740b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 --2- 172.21.15.196:0/2751699077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff5740b0680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5740aff80 tx=0x7ff56c01c9b0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 -- 172.21.15.196:0/2751699077 shutdown_connections 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 --2- 172.21.15.196:0/2751699077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff5740b0680 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 --2- 172.21.15.196:0/2751699077 >> v2:172.21.15.129:3300/0 conn(0x7ff5740cffa0 0x7ff5740d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 -- 172.21.15.196:0/2751699077 >> 172.21.15.196:0/2751699077 conn(0x7ff5740d1230 msgr2=0x7ff5740d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 -- 172.21.15.196:0/2751699077 shutdown_connections 2024-04-25T00:42:35.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.790+0000 7ff57a7df640 1 -- 172.21.15.196:0/2751699077 wait complete. 2024-04-25T00:42:35.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 Processor -- start 2024-04-25T00:42:35.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- start start 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff57406ba90 0x7ff574160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff574153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5740cffa0 0x7ff574162390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff57415c330 con 0x7ff5740cffa0 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff57415c1b0 con 0x7ff5740b01f0 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff5740ce970 con 0x7ff57406ba90 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff574153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff574153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54220/0 (socket says 172.21.15.196:54220) 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff573fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff57406ba90 0x7ff574160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 -- 172.21.15.196:0/796981210 learned_addr learned my addr 172.21.15.196:0/796981210 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff578d7c640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5740cffa0 0x7ff574162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 -- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff57406ba90 msgr2=0x7ff574160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff57406ba90 0x7ff574160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 -- 172.21.15.196:0/796981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5740cffa0 msgr2=0x7ff574162390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5740cffa0 0x7ff574162390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 -- 172.21.15.196:0/796981210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5741612f0 con 0x7ff5740b01f0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff578d7c640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5740cffa0 0x7ff574162390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff573fff640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff57406ba90 0x7ff574160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff56bfff640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff574153420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00d5f0 tx=0x7ff55c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0125f0 con 0x7ff5740b01f0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c019070 con 0x7ff5740b01f0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0129d0 con 0x7ff5740b01f0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741615d0 con 0x7ff5740b01f0 2024-04-25T00:42:35.799 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff574161b90 con 0x7ff5740b01f0 2024-04-25T00:42:35.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff55c012b70 con 0x7ff5740b01f0 2024-04-25T00:42:35.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff55c014a50 con 0x7ff5740b01f0 2024-04-25T00:42:35.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.794+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff574062e50 con 0x7ff5740b01f0 2024-04-25T00:42:35.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.798+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff55c014d30 con 0x7ff5740b01f0 2024-04-25T00:42:35.808 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.806+0000 7f6e11787180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:42:35.833 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.829+0000 7fac369bd180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:42:35.861 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.858+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b19aaa8-4475-4735-a1d5-f2debb8b5165", "id": 5} v 0) -- 0x7ff57415b060 con 0x7ff5740b01f0 2024-04-25T00:42:35.867 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff5717fa640 1 -- 172.21.15.196:0/796981210 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b19aaa8-4475-4735-a1d5-f2debb8b5165", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff55c0144d0 con 0x7ff5740b01f0 2024-04-25T00:42:35.867 INFO:teuthology.orchestra.run.smithi196.stdout:5 2024-04-25T00:42:35.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 msgr2=0x7ff574153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff574153420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00d5f0 tx=0x7ff55c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 shutdown_connections 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5740cffa0 0x7ff574162390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5740b01f0 0x7ff574153420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 --2- 172.21.15.196:0/796981210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff57406ba90 0x7ff574160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 >> 172.21.15.196:0/796981210 conn(0x7ff5740d1230 msgr2=0x7ff57415d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 shutdown_connections 2024-04-25T00:42:35.869 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.866+0000 7ff57a7df640 1 -- 172.21.15.196:0/796981210 wait complete. 2024-04-25T00:42:35.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 5ce24abd-3fc5-415e-a0e2-08a9a816e4c0 6 2024-04-25T00:42:35.887 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.886+0000 7f6e11787180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:42:35.942 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:35.942+0000 7f6e11787180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:42:35.982 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:35.981+0000 7fac369bd180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:42:36.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.998+0000 7fcff116e640 1 Processor -- start 2024-04-25T00:42:36.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:35.998+0000 7fcff116e640 1 -- start start 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcff116e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 0x7fcfec0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcff116e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcfec073090 0x7fcfec15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcff116e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcfec06eb30 0x7fcfec06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcff116e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fcfec06d650 con 0x7fcfec073090 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcff116e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcfec15b330 con 0x7fcfec06eb30 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcff116e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcfec0cd5a0 con 0x7fcfec144130 2024-04-25T00:42:36.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 0x7fcfec0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 0x7fcfec0ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41320/0 (socket says 172.21.15.196:41320) 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfeb59e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcfec06eb30 0x7fcfec06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 -- 172.21.15.196:0/65214916 learned_addr learned my addr 172.21.15.196:0/65214916 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfe3fff640 1 --2- 172.21.15.196:0/65214916 >> v2:172.21.15.129:3300/0 conn(0x7fcfec073090 0x7fcfec15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 -- 172.21.15.196:0/65214916 >> v2:172.21.15.196:3300/0 conn(0x7fcfec06eb30 msgr2=0x7fcfec06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 --2- 172.21.15.196:0/65214916 >> v2:172.21.15.196:3300/0 conn(0x7fcfec06eb30 0x7fcfec06ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 -- 172.21.15.196:0/65214916 >> v2:172.21.15.129:3300/0 conn(0x7fcfec073090 msgr2=0x7fcfec15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 --2- 172.21.15.196:0/65214916 >> v2:172.21.15.129:3300/0 conn(0x7fcfec073090 0x7fcfec15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 -- 172.21.15.196:0/65214916 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec0cd880 con 0x7fcfec144130 2024-04-25T00:42:36.004 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfeb59e640 1 --2- 172.21.15.196:0/65214916 >> v2:172.21.15.196:3300/0 conn(0x7fcfec06eb30 0x7fcfec06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfead9d640 1 --2- 172.21.15.196:0/65214916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 0x7fcfec0ccea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0af9a0 tx=0x7fcfdc0213e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2776ef53c4d9946a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfea59c640 1 -- 172.21.15.196:0/65214916 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0b6070 con 0x7fcfec144130 2024-04-25T00:42:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfea59c640 1 -- 172.21.15.196:0/65214916 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfdc0b1600 con 0x7fcfec144130 2024-04-25T00:42:36.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.002+0000 7fcfea59c640 1 -- 172.21.15.196:0/65214916 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0b19d0 con 0x7fcfec144130 2024-04-25T00:42:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/65214916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 msgr2=0x7fcfec0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- 172.21.15.196:0/65214916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 0x7fcfec0ccea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0af9a0 tx=0x7fcfdc0213e0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/65214916 shutdown_connections 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- 172.21.15.196:0/65214916 >> v2:172.21.15.196:3300/0 conn(0x7fcfec06eb30 0x7fcfec06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- 172.21.15.196:0/65214916 >> v2:172.21.15.129:3300/0 conn(0x7fcfec073090 0x7fcfec15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- 172.21.15.196:0/65214916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec144130 0x7fcfec0ccea0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/65214916 >> 172.21.15.196:0/65214916 conn(0x7fcfec070690 msgr2=0x7fcfec070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/65214916 shutdown_connections 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/65214916 wait complete. 2024-04-25T00:42:36.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 Processor -- start 2024-04-25T00:42:36.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- start start 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfec06eb30 0x7fcfec0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec073090 0x7fcfec0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 0x7fcfec0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0ce900 con 0x7fcfec06eb30 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0ce620 con 0x7fcfec144130 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcfec144510 con 0x7fcfec073090 2024-04-25T00:42:36.010 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec073090 0x7fcfec0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 0x7fcfec0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec073090 0x7fcfec0aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:41322/0 (socket says 172.21.15.196:41322) 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 0x7fcfec0b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54236/0 (socket says 172.21.15.196:54236) 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe3fff640 1 -- 172.21.15.196:0/3899087560 learned_addr learned my addr 172.21.15.196:0/3899087560 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfead9d640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfec06eb30 0x7fcfec0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 -- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec073090 msgr2=0x7fcfec0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec073090 0x7fcfec0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 -- 172.21.15.196:0/3899087560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfec06eb30 msgr2=0x7fcfec0ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfec06eb30 0x7fcfec0ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 -- 172.21.15.196:0/3899087560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec0b04d0 con 0x7fcfec144130 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfeb59e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 0x7fcfec0b0e90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400eed0 tx=0x7fcfe400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4018030 con 0x7fcfec144130 2024-04-25T00:42:36.011 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe400bd20 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec0abb20 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4003bc0 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec0ac3c0 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcfe401b070 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.006+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fcfe40149f0 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.010+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec062e50 con 0x7fcfec144130 2024-04-25T00:42:36.012 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.010+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcfe4022070 con 0x7fcfec144130 2024-04-25T00:42:36.044 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:36.041+0000 7fac369bd180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:42:36.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.074+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5ce24abd-3fc5-415e-a0e2-08a9a816e4c0", "id": 6} v 0) -- 0x7fcfec0aee70 con 0x7fcfec144130 2024-04-25T00:42:36.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcfe37fe640 1 -- 172.21.15.196:0/3899087560 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ce24abd-3fc5-415e-a0e2-08a9a816e4c0", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fcfe40099d0 con 0x7fcfec144130 2024-04-25T00:42:36.086 INFO:teuthology.orchestra.run.smithi196.stdout:6 2024-04-25T00:42:36.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 msgr2=0x7fcfec0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 0x7fcfec0b0e90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400eed0 tx=0x7fcfe400b480 comp rx=0 tx=0).stop 2024-04-25T00:42:36.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 shutdown_connections 2024-04-25T00:42:36.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfec073090 0x7fcfec0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfec144130 0x7fcfec0b0e90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 --2- 172.21.15.196:0/3899087560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfec06eb30 0x7fcfec0ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 >> 172.21.15.196:0/3899087560 conn(0x7fcfec070690 msgr2=0x7fcfec0cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:36.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 shutdown_connections 2024-04-25T00:42:36.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.086+0000 7fcff116e640 1 -- 172.21.15.196:0/3899087560 wait complete. 2024-04-25T00:42:36.099 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:36.098+0000 7f6e11787180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:42:36.102 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new f7c5504f-d630-40a8-a619-c7c8f7c733f8 7 2024-04-25T00:42:36.161 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:36.162+0000 7f6e11787180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:42:36.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.262+0000 7fd289b9e640 1 Processor -- start 2024-04-25T00:42:36.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.262+0000 7fd289b9e640 1 -- start start 2024-04-25T00:42:36.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd289b9e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd284073090 0x7fd284144130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd289b9e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2840b4660 0x7fd28406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd289b9e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2840cd310 con 0x7fd2840ccea0 2024-04-25T00:42:36.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd289b9e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd28406d650 con 0x7fd284073090 2024-04-25T00:42:36.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd289b9e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd2840cd5a0 con 0x7fd2840b4660 2024-04-25T00:42:36.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2840b4660 0x7fd28406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2840b4660 0x7fd28406eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54248/0 (socket says 172.21.15.196:54248) 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 -- 172.21.15.196:0/51161112 learned_addr learned my addr 172.21.15.196:0/51161112 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd2837fe640 1 --2- 172.21.15.196:0/51161112 >> v2:172.21.15.129:3300/0 conn(0x7fd284073090 0x7fd284144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 -- 172.21.15.196:0/51161112 >> v2:172.21.15.129:3300/0 conn(0x7fd284073090 msgr2=0x7fd284144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 --2- 172.21.15.196:0/51161112 >> v2:172.21.15.129:3300/0 conn(0x7fd284073090 0x7fd284144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 -- 172.21.15.196:0/51161112 >> v1:172.21.15.129:6789/0 conn(0x7fd2840ccea0 legacy=0x7fd28415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:36.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 -- 172.21.15.196:0/51161112 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840cd880 con 0x7fd2840b4660 2024-04-25T00:42:36.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd283fff640 1 --2- 172.21.15.196:0/51161112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2840b4660 0x7fd28406eb30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd2840af820 tx=0x7fd27c01f1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d42535f65e1a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:36.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd2827fc640 1 -- 172.21.15.196:0/51161112 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b4070 con 0x7fd2840b4660 2024-04-25T00:42:36.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd2827fc640 1 -- 172.21.15.196:0/51161112 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c0af400 con 0x7fd2840b4660 2024-04-25T00:42:36.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.266+0000 7fd2827fc640 1 -- 172.21.15.196:0/51161112 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0af7d0 con 0x7fd2840b4660 2024-04-25T00:42:36.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/51161112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2840b4660 msgr2=0x7fd28406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 --2- 172.21.15.196:0/51161112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2840b4660 0x7fd28406eb30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd2840af820 tx=0x7fd27c01f1a0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/51161112 shutdown_connections 2024-04-25T00:42:36.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 --2- 172.21.15.196:0/51161112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2840b4660 0x7fd28406eb30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 --2- 172.21.15.196:0/51161112 >> v2:172.21.15.129:3300/0 conn(0x7fd284073090 0x7fd284144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/51161112 >> 172.21.15.196:0/51161112 conn(0x7fd284070690 msgr2=0x7fd284070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:36.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/51161112 shutdown_connections 2024-04-25T00:42:36.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/51161112 wait complete. 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 Processor -- start 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- start start 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 0x7fd284139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2840b4660 0x7fd28413afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2840ccea0 0x7fd284158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840ce900 con 0x7fd2840b4660 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840ce620 con 0x7fd284073090 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd2840b4a70 con 0x7fd2840ccea0 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 0x7fd284139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd283fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2840ccea0 0x7fd284158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 0x7fd284139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54258/0 (socket says 172.21.15.196:54258) 2024-04-25T00:42:36.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 -- 172.21.15.196:0/3058931406 learned_addr learned my addr 172.21.15.196:0/3058931406 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 -- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2840ccea0 msgr2=0x7fd284158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2840ccea0 0x7fd284158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 -- 172.21.15.196:0/3058931406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2840b4660 msgr2=0x7fd28413afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd282ffd640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2840b4660 0x7fd28413afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2840b4660 0x7fd28413afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 -- 172.21.15.196:0/3058931406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd284139f20 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd282ffd640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2840b4660 0x7fd28413afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd2837fe640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 0x7fd284139940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d800 tx=0x7fd27400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400b6c0 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274019070 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400ba60 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28413a200 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28413a7c0 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd27400bc00 con 0x7fd284073090 2024-04-25T00:42:36.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.270+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd274014b30 con 0x7fd284073090 2024-04-25T00:42:36.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.274+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd284062e50 con 0x7fd284073090 2024-04-25T00:42:36.276 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.274+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd274014d30 con 0x7fd284073090 2024-04-25T00:42:36.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.342+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f7c5504f-d630-40a8-a619-c7c8f7c733f8", "id": 7} v 0) -- 0x7fd2840aee70 con 0x7fd284073090 2024-04-25T00:42:36.352 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd26bfff640 1 -- 172.21.15.196:0/3058931406 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f7c5504f-d630-40a8-a619-c7c8f7c733f8", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd27401eb40 con 0x7fd284073090 2024-04-25T00:42:36.352 INFO:teuthology.orchestra.run.smithi196.stdout:7 2024-04-25T00:42:36.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 msgr2=0x7fd284139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 0x7fd284139940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d800 tx=0x7fd27400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 shutdown_connections 2024-04-25T00:42:36.353 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2840ccea0 0x7fd284158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2840b4660 0x7fd28413afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 --2- 172.21.15.196:0/3058931406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd284073090 0x7fd284139940 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 >> 172.21.15.196:0/3058931406 conn(0x7fd284070690 msgr2=0x7fd28406eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 shutdown_connections 2024-04-25T00:42:36.354 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:42:36.350+0000 7fd289b9e640 1 -- 172.21.15.196:0/3058931406 wait complete. 2024-04-25T00:42:36.369 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T00:42:36.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T00:42:36.371 INFO:tasks.ceph.osd.0:Started 2024-04-25T00:42:36.372 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T00:42:36.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T00:42:36.374 INFO:tasks.ceph.osd.1:Started 2024-04-25T00:42:36.374 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T00:42:36.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T00:42:36.376 INFO:tasks.ceph.osd.2:Started 2024-04-25T00:42:36.377 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T00:42:36.377 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T00:42:36.379 INFO:tasks.ceph.osd.3:Started 2024-04-25T00:42:36.379 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T00:42:36.379 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T00:42:36.410 INFO:tasks.ceph.osd.4:Started 2024-04-25T00:42:36.411 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T00:42:36.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T00:42:36.413 INFO:tasks.ceph.osd.5:Started 2024-04-25T00:42:36.413 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T00:42:36.413 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T00:42:36.415 INFO:tasks.ceph.osd.6:Started 2024-04-25T00:42:36.415 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T00:42:36.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T00:42:36.419 INFO:tasks.ceph.osd.7:Started 2024-04-25T00:42:36.419 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:36.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 Processor -- start 2024-04-25T00:42:36.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 -- start start 2024-04-25T00:42:36.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fde840cf270 0x7fde840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde84147f20 0x7fde84070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8406b4b0 con 0x7fde84147b40 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fde8406b770 con 0x7fde840cf270 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8bc63640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde8406ba50 con 0x7fde84147f20 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde891fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fde840cf270 0x7fde840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde899ff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fde84147b40 0x7fde840cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:48540/0 (socket says 172.21.15.129:48540) 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde899ff640 1 -- 172.21.15.129:0/1568832470 learned_addr learned my addr 172.21.15.129:0/1568832470 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde8a200640 1 --2- 172.21.15.129:0/1568832470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde84147f20 0x7fde84070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde891fe640 1 -- 172.21.15.129:0/1568832470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde84147f20 msgr2=0x7fde84070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde891fe640 1 --2- 172.21.15.129:0/1568832470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde84147f20 0x7fde84070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde891fe640 1 -- 172.21.15.129:0/1568832470 >> v1:172.21.15.129:6789/0 conn(0x7fde84147b40 legacy=0x7fde840cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.553+0000 7fde891fe640 1 -- 172.21.15.129:0/1568832470 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8406bd30 con 0x7fde840cf270 2024-04-25T00:42:36.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde891fe640 1 --2- 172.21.15.129:0/1568832470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde840cf270 0x7fde840706d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde740097c0 tx=0x7fde7401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45b05acd63a4343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:36.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde889fd640 1 -- 172.21.15.129:0/1568832470 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740b0070 con 0x7fde840cf270 2024-04-25T00:42:36.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde889fd640 1 -- 172.21.15.129:0/1568832470 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde740ab600 con 0x7fde840cf270 2024-04-25T00:42:36.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde889fd640 1 -- 172.21.15.129:0/1568832470 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740ab9d0 con 0x7fde840cf270 2024-04-25T00:42:36.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 -- 172.21.15.129:0/1568832470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde840cf270 msgr2=0x7fde840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 --2- 172.21.15.129:0/1568832470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde840cf270 0x7fde840706d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde740097c0 tx=0x7fde7401c410 comp rx=0 tx=0).stop 2024-04-25T00:42:36.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 -- 172.21.15.129:0/1568832470 shutdown_connections 2024-04-25T00:42:36.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 --2- 172.21.15.129:0/1568832470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde84147f20 0x7fde84070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 --2- 172.21.15.129:0/1568832470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde840cf270 0x7fde840706d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 -- 172.21.15.129:0/1568832470 >> 172.21.15.129:0/1568832470 conn(0x7fde840b2380 msgr2=0x7fde840b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:36.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 -- 172.21.15.129:0/1568832470 shutdown_connections 2024-04-25T00:42:36.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 -- 172.21.15.129:0/1568832470 wait complete. 2024-04-25T00:42:36.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 Processor -- start 2024-04-25T00:42:36.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.557+0000 7fde8bc63640 1 -- start start 2024-04-25T00:42:36.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde840cf270 0x7fde84073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 0x7fde8413dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde84147f20 0x7fde8413be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde840ccac0 con 0x7fde84147b40 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde840cc940 con 0x7fde84147f20 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde840ccc40 con 0x7fde840cf270 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 0x7fde8413dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 0x7fde8413dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59186/0 (socket says 172.21.15.129:59186) 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 -- 172.21.15.129:0/2878860018 learned_addr learned my addr 172.21.15.129:0/2878860018 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 -- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde840cf270 msgr2=0x7fde84073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde840cf270 0x7fde84073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 -- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde84147f20 msgr2=0x7fde8413be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8a200640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde84147f20 0x7fde8413be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde84147f20 0x7fde8413be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 -- 172.21.15.129:0/2878860018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8413d240 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde891fe640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 0x7fde8413dae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fde7401a040 tx=0x7fde740191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740b8070 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde740abe10 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8413d520 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740b0040 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde841588c0 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde740bb070 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fde740bdd50 con 0x7fde84147b40 2024-04-25T00:42:36.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.561+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde84062e50 con 0x7fde84147b40 2024-04-25T00:42:36.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.565+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fde74016090 con 0x7fde84147b40 2024-04-25T00:42:36.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.617+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde84075320 con 0x7fde84147b40 2024-04-25T00:42:36.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde7a7fc640 1 -- 172.21.15.129:0/2878860018 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fde740aa9f0 con 0x7fde84147b40 2024-04-25T00:42:36.622 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:36.622 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":10,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:36.346639+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:36.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 msgr2=0x7fde8413dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 0x7fde8413dae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fde7401a040 tx=0x7fde740191b0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 shutdown_connections 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde84147f20 0x7fde8413be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde84147b40 0x7fde8413dae0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 --2- 172.21.15.129:0/2878860018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde840cf270 0x7fde84073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 >> 172.21.15.129:0/2878860018 conn(0x7fde840b2380 msgr2=0x7fde84157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 shutdown_connections 2024-04-25T00:42:36.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:36.621+0000 7fde8bc63640 1 -- 172.21.15.129:0/2878860018 wait complete. 2024-04-25T00:42:36.638 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T00:42:36.638 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T00:42:36.686 INFO:tasks.ceph.mgr.y.smithi129.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:42:36.686 INFO:tasks.ceph.mgr.y.smithi129.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:42:36.686 INFO:tasks.ceph.mgr.y.smithi129.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:42:36.732 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:36.729+0000 7fac369bd180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:42:36.827 INFO:tasks.ceph.mgr.x.smithi196.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:42:36.827 INFO:tasks.ceph.mgr.x.smithi196.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:42:36.827 INFO:tasks.ceph.mgr.x.smithi196.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:42:36.866 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:36.861+0000 7fac369bd180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:42:36.888 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:36.886+0000 7f6e11787180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:42:37.024 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:37.021+0000 7fac369bd180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:42:37.027 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T00:42:37.026+0000 7f4f0b40b7c0 -1 Falling back to public interface 2024-04-25T00:42:37.028 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T00:42:37.026+0000 7fd140fbf7c0 -1 Falling back to public interface 2024-04-25T00:42:37.028 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T00:42:37.026+0000 7f1f0a55c7c0 -1 Falling back to public interface 2024-04-25T00:42:37.039 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T00:42:37.037+0000 7f79f02a67c0 -1 Falling back to public interface 2024-04-25T00:42:37.040 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T00:42:37.037+0000 7f89647087c0 -1 Falling back to public interface 2024-04-25T00:42:37.040 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T00:42:37.037+0000 7f79ef266640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6802/0: (98) Address already in use 2024-04-25T00:42:37.040 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T00:42:37.037+0000 7fe19584a7c0 -1 Falling back to public interface 2024-04-25T00:42:37.046 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T00:42:37.046+0000 7f26471ac7c0 -1 Falling back to public interface 2024-04-25T00:42:37.047 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T00:42:37.045+0000 7f4d41fa27c0 -1 Falling back to public interface 2024-04-25T00:42:37.065 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:37.062+0000 7f6e11787180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:42:37.093 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:37.089+0000 7fac369bd180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:42:37.217 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:37.218+0000 7f6e11787180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:42:37.227 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:37.225+0000 7fac369bd180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:42:37.281 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:37.278+0000 7f6e11787180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:42:37.415 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:37.414+0000 7f6e11787180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:42:37.442 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:37.557 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.553+0000 7f33a7cba640 1 Processor -- start 2024-04-25T00:42:37.558 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.553+0000 7f33a7cba640 1 -- start start 2024-04-25T00:42:37.559 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a7cba640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f33a015ac10 0x7f33a006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a7cba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a006cc00 0x7f33a00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a7cba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33a00b4fe0 0x7f33a0071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a7cba640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f33a015e9a0 con 0x7f33a015ac10 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a7cba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f33a015ec80 con 0x7f33a00b4fe0 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a7cba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f33a015ef60 con 0x7f33a006cc00 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f33a015ac10 0x7f33a006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f33a015ac10 0x7f33a006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59198/0 (socket says 172.21.15.129:59198) 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 -- 172.21.15.129:0/105938346 learned_addr learned my addr 172.21.15.129:0/105938346 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5255640 1 --2- 172.21.15.129:0/105938346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a006cc00 0x7f33a00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a6257640 1 --2- 172.21.15.129:0/105938346 >> v2:172.21.15.196:3300/0 conn(0x7f33a00b4fe0 0x7f33a0071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 -- 172.21.15.129:0/105938346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a006cc00 msgr2=0x7f33a00b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 --2- 172.21.15.129:0/105938346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a006cc00 0x7f33a00b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 -- 172.21.15.129:0/105938346 >> v2:172.21.15.196:3300/0 conn(0x7f33a00b4fe0 msgr2=0x7f33a0071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.560 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 --2- 172.21.15.129:0/105938346 >> v2:172.21.15.196:3300/0 conn(0x7f33a00b4fe0 0x7f33a0071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.561 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 -- 172.21.15.129:0/105938346 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a015f240 con 0x7f33a015ac10 2024-04-25T00:42:37.561 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a5a56640 1 --2- 172.21.15.129:0/105938346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a006c6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33a006e1c0 tx=0x7f339c01e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f863820ab852c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.561 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a4a54640 1 -- 172.21.15.129:0/105938346 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0b4070 con 0x7f33a015ac10 2024-04-25T00:42:37.561 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a4a54640 1 -- 172.21.15.129:0/105938346 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c0a7730 con 0x7f33a015ac10 2024-04-25T00:42:37.562 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.557+0000 7f33a4a54640 1 -- 172.21.15.129:0/105938346 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0a7b00 con 0x7f33a015ac10 2024-04-25T00:42:37.562 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/105938346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 msgr2=0x7f33a006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- 172.21.15.129:0/105938346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a006c6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33a006e1c0 tx=0x7f339c01e8a0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/105938346 shutdown_connections 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- 172.21.15.129:0/105938346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a006cc00 0x7f33a00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- 172.21.15.129:0/105938346 >> v2:172.21.15.196:3300/0 conn(0x7f33a00b4fe0 0x7f33a0071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- 172.21.15.129:0/105938346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a006c6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/105938346 >> 172.21.15.129:0/105938346 conn(0x7f33a00b2a80 msgr2=0x7f33a00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/105938346 shutdown_connections 2024-04-25T00:42:37.563 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/105938346 wait complete. 2024-04-25T00:42:37.566 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 Processor -- start 2024-04-25T00:42:37.566 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- start start 2024-04-25T00:42:37.566 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a006cc00 0x7f33a014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.566 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a00b4fe0 0x7f33a013b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.566 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.566 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33a006b390 con 0x7f33a015ac10 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33a006b210 con 0x7f33a006cc00 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f33a006b510 con 0x7f33a00b4fe0 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a014df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59202/0 (socket says 172.21.15.129:59202) 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 -- 172.21.15.129:0/3234294396 learned_addr learned my addr 172.21.15.129:0/3234294396 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a5255640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a00b4fe0 0x7f33a013b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a5a56640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a006cc00 0x7f33a014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 -- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a00b4fe0 msgr2=0x7f33a013b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a00b4fe0 0x7f33a013b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 -- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a006cc00 msgr2=0x7f33a014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a006cc00 0x7f33a014c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 -- 172.21.15.129:0/3234294396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a014c8a0 con 0x7f33a015ac10 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a5a56640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a006cc00 0x7f33a014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:37.567 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a6257640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a014df90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f339800cd70 tx=0x7f339800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398003ba0 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3398003d40 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a014cb80 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004240 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01456f0 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f339801f440 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3398016ba0 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.561+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a01458f0 con 0x7f33a015ac10 2024-04-25T00:42:37.568 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.565+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f33a01458f0 con 0x7f33a015ac10 2024-04-25T00:42:37.625 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.621+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33a014d0a0 con 0x7f33a015ac10 2024-04-25T00:42:37.625 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.621+0000 7f33967fc640 1 -- 172.21.15.129:0/3234294396 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f339801f910 con 0x7f33a015ac10 2024-04-25T00:42:37.625 INFO:teuthology.misc.health.smithi129.stdout: 2024-04-25T00:42:37.626 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":10,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:36.346639+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:37.626 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 msgr2=0x7f33a014df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a014df90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f339800cd70 tx=0x7f339800a660 comp rx=0 tx=0).stop 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 shutdown_connections 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33a00b4fe0 0x7f33a013b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33a015ac10 0x7f33a014df90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 --2- 172.21.15.129:0/3234294396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a006cc00 0x7f33a014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 >> 172.21.15.129:0/3234294396 conn(0x7f33a00b2a80 msgr2=0x7f33a006bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 shutdown_connections 2024-04-25T00:42:37.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:37.625+0000 7f33a7cba640 1 -- 172.21.15.129:0/3234294396 wait complete. 2024-04-25T00:42:37.642 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:42:37.886 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:37.885+0000 7fac369bd180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:42:37.963 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:37.961+0000 7fac369bd180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:42:38.041 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:38.038+0000 7f6e11787180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:42:38.116 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:38.114+0000 7f6e11787180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:42:38.146 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:38.141+0000 7fac369bd180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:42:38.219 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:38.217+0000 7fac369bd180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:42:38.297 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:38.294+0000 7f6e11787180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:42:38.370 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:38.370+0000 7f6e11787180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:42:38.509 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:38.505+0000 7fac369bd180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:42:38.571 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:38.569+0000 7fac369bd180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:42:38.665 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:38.662+0000 7f6e11787180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:42:38.726 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T00:42:38.726+0000 7f6e11787180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:42:39.619 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:39.617+0000 7fabdde09640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T00:42:39.619 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T00:42:39.617+0000 7fabdde09640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T00:42:40.877 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T00:42:40.878+0000 7f4f0b40b7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T00:42:40.896 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T00:42:40.893+0000 7f79f02a67c0 -1 osd.0 0 log_to_monitors true 2024-04-25T00:42:40.901 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T00:42:40.897+0000 7f89647087c0 -1 osd.1 0 log_to_monitors true 2024-04-25T00:42:40.903 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T00:42:40.902+0000 7fd140fbf7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T00:42:40.914 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T00:42:40.913+0000 7f4d41fa27c0 -1 osd.2 0 log_to_monitors true 2024-04-25T00:42:40.924 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T00:42:40.922+0000 7f26471ac7c0 -1 osd.5 0 log_to_monitors true 2024-04-25T00:42:40.929 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T00:42:40.925+0000 7fe19584a7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T00:42:40.948 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T00:42:40.946+0000 7f1f0a55c7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T00:42:44.449 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:44.622 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.617+0000 7f790efd4640 1 Processor -- start 2024-04-25T00:42:44.622 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f790efd4640 1 -- start start 2024-04-25T00:42:44.624 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f790efd4640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f79080d1630 0x7f7908166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.624 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f790efd4640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79080ccd70 con 0x7f79080c73b0 2024-04-25T00:42:44.624 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f790efd4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79080cd030 con 0x7f790815c210 2024-04-25T00:42:44.624 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f790efd4640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f79080cd960 con 0x7f79080d1630 2024-04-25T00:42:44.625 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f7907fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f790815c210 0x7f79080cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:50602/0 (socket says 172.21.15.129:50602) 2024-04-25T00:42:44.625 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f7907fff640 1 -- 172.21.15.129:0/2222256905 learned_addr learned my addr 172.21.15.129:0/2222256905 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:44.625 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f790d571640 1 --2- 172.21.15.129:0/2222256905 >> v2:172.21.15.129:3300/0 conn(0x7f79080d1630 0x7f7908166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.626 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355904235 0 0) 0x7f79080cd030 con 0x7f790815c210 2024-04-25T00:42:44.626 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f0003530 con 0x7f790815c210 2024-04-25T00:42:44.626 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938537465 0 0) 0x7f79080ccd70 con 0x7f79080c73b0 2024-04-25T00:42:44.626 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.621+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79080cd030 con 0x7f79080c73b0 2024-04-25T00:42:44.626 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.625+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 833051137 0 0) 0x7f79080cd030 con 0x7f79080c73b0 2024-04-25T00:42:44.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.625+0000 7f790d571640 1 -- 172.21.15.129:0/2222256905 >> v1:172.21.15.196:6789/0 conn(0x7f790815c210 legacy=0x7f79080cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.625+0000 7f790d571640 1 -- 172.21.15.129:0/2222256905 >> v1:172.21.15.129:6789/0 conn(0x7f79080c73b0 legacy=0x7f79080c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.627 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.625+0000 7f790d571640 1 -- 172.21.15.129:0/2222256905 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79080cd310 con 0x7f79080d1630 2024-04-25T00:42:44.630 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790d571640 1 --2- 172.21.15.129:0/2222256905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f7908166b30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009740 tx=0x7f78fc01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2448c107b7fa1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:44.631 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0b0070 con 0x7f79080d1630 2024-04-25T00:42:44.631 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78fc0ab400 con 0x7f79080d1630 2024-04-25T00:42:44.631 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f79077fe640 1 -- 172.21.15.129:0/2222256905 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000018b0 con 0x7f79080d1630 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 -- 172.21.15.129:0/2222256905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 msgr2=0x7f7908166b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 --2- 172.21.15.129:0/2222256905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f7908166b30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009740 tx=0x7f78fc01c2d0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 -- 172.21.15.129:0/2222256905 shutdown_connections 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 --2- 172.21.15.129:0/2222256905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f7908166b30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 -- 172.21.15.129:0/2222256905 >> 172.21.15.129:0/2222256905 conn(0x7f79080d1d00 msgr2=0x7f79080d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 -- 172.21.15.129:0/2222256905 shutdown_connections 2024-04-25T00:42:44.633 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.629+0000 7f790efd4640 1 -- 172.21.15.129:0/2222256905 wait complete. 2024-04-25T00:42:44.637 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.633+0000 7f790efd4640 1 Processor -- start 2024-04-25T00:42:44.637 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.633+0000 7f790efd4640 1 -- start start 2024-04-25T00:42:44.639 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790efd4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79080c73b0 0x7f79081465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.639 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790efd4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f790806dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790efd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f790815c210 0x7f79080b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790efd4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7908169c20 con 0x7f79080d1630 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790efd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7908169aa0 con 0x7f790815c210 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790efd4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7908169da0 con 0x7f79080c73b0 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f790806dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f790806dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38846/0 (socket says 172.21.15.129:38846) 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 -- 172.21.15.129:0/2462324346 learned_addr learned my addr 172.21.15.129:0/2462324346 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790cd70640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79080c73b0 0x7f79081465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f790d571640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f790815c210 0x7f79080b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 -- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79080c73b0 msgr2=0x7f79081465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79080c73b0 0x7f79081465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 -- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f790815c210 msgr2=0x7f79080b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f790815c210 0x7f79080b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.640 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 -- 172.21.15.129:0/2462324346 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790806cd30 con 0x7f79080d1630 2024-04-25T00:42:44.641 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.637+0000 7f7907fff640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f790806dba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78f800cac0 tx=0x7f78f800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:44.642 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.641+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8018070 con 0x7f79080d1630 2024-04-25T00:42:44.642 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.641+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f80074e0 con 0x7f79080d1630 2024-04-25T00:42:44.642 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.641+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790815de90 con 0x7f79080d1630 2024-04-25T00:42:44.642 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.641+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80049f0 con 0x7f79080d1630 2024-04-25T00:42:44.643 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.641+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790806d2b0 con 0x7f79080d1630 2024-04-25T00:42:44.643 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.641+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7908068ba0 con 0x7f79080d1630 2024-04-25T00:42:44.646 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.645+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f78f801b070 con 0x7f79080d1630 2024-04-25T00:42:44.650 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.645+0000 7f79057fa640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78f4067900 0x7f78f4069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:44.650 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.645+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f78f8084e30 con 0x7f79080d1630 2024-04-25T00:42:44.650 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.649+0000 7f790cd70640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78f4067900 0x7f78f4069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:44.652 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.649+0000 7f790cd70640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78f4067900 0x7f78f4069dc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7900007fd0 tx=0x7f7900009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:44.658 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.653+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78f8056170 con 0x7f79080d1630 2024-04-25T00:42:44.673 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T00:42:44.669+0000 7f2642932640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:42:44.698 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T00:42:44.697+0000 7f263af79640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.760 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T00:42:44.757+0000 7fd13c745640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:42:44.782 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T00:42:44.781+0000 7fd134d8c640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.790 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T00:42:44.789+0000 7f1f05ce2640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:42:44.791 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T00:42:44.789+0000 7f4f083c7640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:42:44.802 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T00:42:44.801+0000 7f1efe329640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.806 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T00:42:44.805+0000 7f4eff1d8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.812 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T00:42:44.809+0000 7fe190fd0640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:42:44.835 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T00:42:44.833+0000 7fe189617640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.852 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T00:42:44.849+0000 7f895fe8e640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:42:44.868 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T00:42:44.865+0000 7f89584d5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.868 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T00:42:44.865+0000 7f4d3d728640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:42:44.874 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T00:42:44.873+0000 7f79eba2c640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:42:44.903 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T00:42:44.901+0000 7f79e4073640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.903 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T00:42:44.901+0000 7f4d35d6f640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:42:44.925 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.921+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79080b5820 con 0x7f79080d1630 2024-04-25T00:42:44.926 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.921+0000 7f79057fa640 1 -- 172.21.15.129:0/2462324346 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f78f805a470 con 0x7f79080d1630 2024-04-25T00:42:44.926 INFO:teuthology.misc.health.smithi129.stdout: 2024-04-25T00:42:44.926 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":12,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:42.620586+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:44.928 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78f4067900 msgr2=0x7f78f4069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78f4067900 0x7f78f4069dc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7900007fd0 tx=0x7f7900009040 comp rx=0 tx=0).stop 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 msgr2=0x7f790806dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f790806dba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78f800cac0 tx=0x7f78f800cf80 comp rx=0 tx=0).stop 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 shutdown_connections 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78f4067900 0x7f78f4069dc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f790815c210 0x7f79080b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79080d1630 0x7f790806dba0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 --2- 172.21.15.129:0/2462324346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79080c73b0 0x7f79081465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 >> 172.21.15.129:0/2462324346 conn(0x7f79080d1d00 msgr2=0x7f790806ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 shutdown_connections 2024-04-25T00:42:44.929 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:44.925+0000 7f790efd4640 1 -- 172.21.15.129:0/2462324346 wait complete. 2024-04-25T00:42:44.943 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:42:51.750 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:51.871 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 Processor -- start 2024-04-25T00:42:51.871 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 -- start start 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f77501617d0 0x7f77500b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f77500b6e30 0x7f77500d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77500b54a0 con 0x7f77500d2860 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f77500d1bb0 con 0x7f77500b6e30 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f7754a62640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f77500d1e90 con 0x7f77501617d0 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f77500b6e30 0x7f77500d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f77500b6e30 0x7f77500d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38864/0 (socket says 172.21.15.129:38864) 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 -- 172.21.15.129:0/2233142035 learned_addr learned my addr 172.21.15.129:0/2233142035 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774e59c640 1 --2- 172.21.15.129:0/2233142035 >> v2:172.21.15.196:3300/0 conn(0x7f77501617d0 0x7f77500b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 -- 172.21.15.129:0/2233142035 >> v2:172.21.15.196:3300/0 conn(0x7f77501617d0 msgr2=0x7f77500b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 --2- 172.21.15.129:0/2233142035 >> v2:172.21.15.196:3300/0 conn(0x7f77501617d0 0x7f77500b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 -- 172.21.15.129:0/2233142035 >> v1:172.21.15.129:6789/0 conn(0x7f77500d2860 legacy=0x7f77500d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774d59a640 1 -- 172.21.15.129:0/2233142035 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844136273 0 0) 0x7f77480014a0 con 0x7f77500d2860 2024-04-25T00:42:51.873 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.868+0000 7f774dd9b640 1 -- 172.21.15.129:0/2233142035 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77500b72b0 con 0x7f77500b6e30 2024-04-25T00:42:51.875 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774dd9b640 1 --2- 172.21.15.129:0/2233142035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77500b6e30 0x7f77500d2320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f775008d8e0 tx=0x7f774401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f127b3e7068cff46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.875 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774d59a640 1 -- 172.21.15.129:0/2233142035 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440ab600 con 0x7f77500b6e30 2024-04-25T00:42:51.875 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774d59a640 1 -- 172.21.15.129:0/2233142035 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77440b0070 con 0x7f77500b6e30 2024-04-25T00:42:51.875 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774d59a640 1 -- 172.21.15.129:0/2233142035 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440ab9c0 con 0x7f77500b6e30 2024-04-25T00:42:51.876 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- 172.21.15.129:0/2233142035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77500b6e30 msgr2=0x7f77500d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.876 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 --2- 172.21.15.129:0/2233142035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77500b6e30 0x7f77500d2320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f775008d8e0 tx=0x7f774401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.876 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- 172.21.15.129:0/2233142035 shutdown_connections 2024-04-25T00:42:51.876 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 --2- 172.21.15.129:0/2233142035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77500b6e30 0x7f77500d2320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.877 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 --2- 172.21.15.129:0/2233142035 >> v2:172.21.15.196:3300/0 conn(0x7f77501617d0 0x7f77500b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.877 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- 172.21.15.129:0/2233142035 >> 172.21.15.129:0/2233142035 conn(0x7f77500d3fe0 msgr2=0x7f77500d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.877 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- 172.21.15.129:0/2233142035 shutdown_connections 2024-04-25T00:42:51.877 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- 172.21.15.129:0/2233142035 wait complete. 2024-04-25T00:42:51.877 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 Processor -- start 2024-04-25T00:42:51.877 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- start start 2024-04-25T00:42:51.879 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77500b6e30 0x7f775014b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.879 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77500d2860 0x7f775014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.879 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 0x7f775014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f775016b230 con 0x7f77501617d0 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7750161bb0 con 0x7f77500d2860 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f7754a62640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f775016b3b0 con 0x7f77500b6e30 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 0x7f775014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 0x7f775014c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38870/0 (socket says 172.21.15.129:38870) 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.872+0000 7f774ed9d640 1 -- 172.21.15.129:0/3332440701 learned_addr learned my addr 172.21.15.129:0/3332440701 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774e59c640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77500b6e30 0x7f775014b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774dd9b640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77500d2860 0x7f775014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774ed9d640 1 -- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77500b6e30 msgr2=0x7f775014b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774ed9d640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77500b6e30 0x7f775014b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774ed9d640 1 -- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77500d2860 msgr2=0x7f775014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774ed9d640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77500d2860 0x7f775014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.880 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774ed9d640 1 -- 172.21.15.129:0/3332440701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7750154b90 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774ed9d640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 0x7f775014c750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f774800c7b0 tx=0x7f774800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748003db0 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748005070 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748005460 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7750154e70 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7750155430 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7750068ba0 con 0x7f77501617d0 2024-04-25T00:42:51.881 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7748005600 con 0x7f77501617d0 2024-04-25T00:42:51.882 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f773effd640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f77300678c0 0x7f7730069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.882 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774e59c640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f77300678c0 0x7f7730069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.882 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7748085860 con 0x7f77501617d0 2024-04-25T00:42:51.882 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.876+0000 7f774e59c640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f77300678c0 0x7f7730069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f774000b930 tx=0x7f7740008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.885 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:51.880+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77500b54a0 con 0x7f77501617d0 2024-04-25T00:42:52.043 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.040+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f775006b470 con 0x7f77501617d0 2024-04-25T00:42:52.044 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.040+0000 7f773effd640 1 -- 172.21.15.129:0/3332440701 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f7748056780 con 0x7f77501617d0 2024-04-25T00:42:52.045 INFO:teuthology.misc.health.smithi129.stdout: 2024-04-25T00:42:52.045 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":16,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:48.886021+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f77300678c0 msgr2=0x7f7730069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f77300678c0 0x7f7730069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f774000b930 tx=0x7f7740008040 comp rx=0 tx=0).stop 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 msgr2=0x7f775014c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 0x7f775014c750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f774800c7b0 tx=0x7f774800cc70 comp rx=0 tx=0).stop 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 shutdown_connections 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f77300678c0 0x7f7730069d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77501617d0 0x7f775014c750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77500d2860 0x7f775014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 --2- 172.21.15.129:0/3332440701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77500b6e30 0x7f775014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.047 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 >> 172.21.15.129:0/3332440701 conn(0x7f77500d3fe0 msgr2=0x7f775015ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.048 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 shutdown_connections 2024-04-25T00:42:52.048 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T00:42:52.044+0000 7f7754a62640 1 -- 172.21.15.129:0/3332440701 wait complete. 2024-04-25T00:42:52.061 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T00:42:52.061 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T00:42:52.062 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:42:52.062 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:42:52.064 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:42:52.064 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T00:42:52.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T00:42:52.066 INFO:tasks.ceph.mds.c:Started 2024-04-25T00:42:52.066 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:42:52.066 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:42:52.069 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:42:52.069 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T00:42:52.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T00:42:52.072 INFO:tasks.ceph.mds.d:Started 2024-04-25T00:42:52.072 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T00:42:52.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:52.128 INFO:tasks.ceph.mds.c.smithi129.stdout:starting mds.c at 2024-04-25T00:42:52.129 INFO:tasks.ceph.mds.a.smithi129.stdout:starting mds.a at 2024-04-25T00:42:52.149 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T00:42:52.152 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T00:42:52.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 Processor -- start 2024-04-25T00:42:52.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 -- start start 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f825c1441b0 0x7f825c1627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f825c144590 0x7f825c0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f825c162cf0 con 0x7f825c0b6a20 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f825c0b6e80 con 0x7f825c144590 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8262e49640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f825c15f6d0 con 0x7f825c1441b0 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f825c144590 0x7f825c0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f825c144590 0x7f825c0b64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38920/0 (socket says 172.21.15.129:38920) 2024-04-25T00:42:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 -- 172.21.15.129:0/937134696 learned_addr learned my addr 172.21.15.129:0/937134696 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f8260be5640 1 --2- 172.21.15.129:0/937134696 >> v2:172.21.15.196:3300/0 conn(0x7f825c1441b0 0x7f825c1627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 -- 172.21.15.129:0/937134696 >> v2:172.21.15.196:3300/0 conn(0x7f825c1441b0 msgr2=0x7f825c1627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 --2- 172.21.15.129:0/937134696 >> v2:172.21.15.196:3300/0 conn(0x7f825c1441b0 0x7f825c1627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 -- 172.21.15.129:0/937134696 >> v1:172.21.15.196:6789/0 conn(0x7f825c0b6a20 legacy=0x7f825c071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.180+0000 7f825bfff640 1 -- 172.21.15.129:0/937134696 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c15eac0 con 0x7f825c144590 2024-04-25T00:42:52.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f825bfff640 1 --2- 172.21.15.129:0/937134696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c144590 0x7f825c0b64e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f825c070d70 tx=0x7f824801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb37aee41c9a48c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f825b7fe640 1 -- 172.21.15.129:0/937134696 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480aa650 con 0x7f825c144590 2024-04-25T00:42:52.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f825b7fe640 1 -- 172.21.15.129:0/937134696 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82480af070 con 0x7f825c144590 2024-04-25T00:42:52.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f825b7fe640 1 -- 172.21.15.129:0/937134696 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480aaa10 con 0x7f825c144590 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 -- 172.21.15.129:0/937134696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c144590 msgr2=0x7f825c0b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 --2- 172.21.15.129:0/937134696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c144590 0x7f825c0b64e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f825c070d70 tx=0x7f824801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 -- 172.21.15.129:0/937134696 shutdown_connections 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 --2- 172.21.15.129:0/937134696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c144590 0x7f825c0b64e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 --2- 172.21.15.129:0/937134696 >> v2:172.21.15.196:3300/0 conn(0x7f825c1441b0 0x7f825c1627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 -- 172.21.15.129:0/937134696 >> 172.21.15.129:0/937134696 conn(0x7f825c0b52d0 msgr2=0x7f825c0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 -- 172.21.15.129:0/937134696 shutdown_connections 2024-04-25T00:42:52.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 -- 172.21.15.129:0/937134696 wait complete. 2024-04-25T00:42:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 Processor -- start 2024-04-25T00:42:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.184+0000 7f8262e49640 1 -- start start 2024-04-25T00:42:52.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f825c0b6a20 0x7f825c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 0x7f825c144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f825c144590 0x7f825c144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f825c143a90 con 0x7f825c1441b0 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f825c15e760 con 0x7f825c0b6a20 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f825c143c10 con 0x7f825c144590 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 0x7f825c144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 0x7f825c144a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38928/0 (socket says 172.21.15.129:38928) 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 -- 172.21.15.129:0/2862058858 learned_addr learned my addr 172.21.15.129:0/2862058858 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8260be5640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f825c0b6a20 0x7f825c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82613e6640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f825c144590 0x7f825c144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 -- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f825c144590 msgr2=0x7f825c144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f825c144590 0x7f825c144f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 -- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f825c0b6a20 msgr2=0x7f825c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f825c0b6a20 0x7f825c158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 -- 172.21.15.129:0/2862058858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1593b0 con 0x7f825c1441b0 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8260be5640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f825c0b6a20 0x7f825c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f825bfff640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 0x7f825c144a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8248019100 tx=0x7f8248019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480be610 con 0x7f825c1441b0 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82480a6040 con 0x7f825c1441b0 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c13f2e0 con 0x7f825c1441b0 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480be9f0 con 0x7f825c1441b0 2024-04-25T00:42:52.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c154640 con 0x7f825c1441b0 2024-04-25T00:42:52.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c062e50 con 0x7f825c1441b0 2024-04-25T00:42:52.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f82480bc020 con 0x7f825c1441b0 2024-04-25T00:42:52.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82597fa640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f82400678c0 0x7f8240069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f82481203b0 con 0x7f825c1441b0 2024-04-25T00:42:52.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8260be5640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f82400678c0 0x7f8240069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.188+0000 7f8260be5640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f82400678c0 0x7f8240069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8250006aa0 tx=0x7f8250008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.192+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f82480c3060 con 0x7f825c1441b0 2024-04-25T00:42:52.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.348+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f825c071230 con 0x7f825c1441b0 2024-04-25T00:42:52.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f82597fa640 1 -- 172.21.15.129:0/2862058858 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f8248016110 con 0x7f825c1441b0 2024-04-25T00:42:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:52.356 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":16,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:48.886021+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:52.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f82400678c0 msgr2=0x7f8240069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f82400678c0 0x7f8240069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8250006aa0 tx=0x7f8250008040 comp rx=0 tx=0).stop 2024-04-25T00:42:52.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 msgr2=0x7f825c144a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 0x7f825c144a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8248019100 tx=0x7f8248019290 comp rx=0 tx=0).stop 2024-04-25T00:42:52.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 shutdown_connections 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f82400678c0 0x7f8240069d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f825c144590 0x7f825c144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f825c1441b0 0x7f825c144a20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 --2- 172.21.15.129:0/2862058858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f825c0b6a20 0x7f825c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 >> 172.21.15.129:0/2862058858 conn(0x7f825c0b52d0 msgr2=0x7f825c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 shutdown_connections 2024-04-25T00:42:52.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.352+0000 7f8262e49640 1 -- 172.21.15.129:0/2862058858 wait complete. 2024-04-25T00:42:52.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:42:52.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:42:52.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa713d4640 1 Processor -- start 2024-04-25T00:42:52.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa713d4640 1 -- start start 2024-04-25T00:42:52.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa713d4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa6c06eba0 0x7faa6c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa713d4640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6c06f110 con 0x7faa6c0ce190 2024-04-25T00:42:52.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa713d4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6c06f3f0 con 0x7faa6c06cc40 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa713d4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faa6c06f6d0 con 0x7faa6c06eba0 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6b7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7faa6c0ce190 0x7faa6c0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46836/0 (socket says 172.21.15.129:46836) 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6b7fe640 1 -- 172.21.15.129:0/1921538028 learned_addr learned my addr 172.21.15.129:0/1921538028 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 --2- 172.21.15.129:0/1921538028 >> v2:172.21.15.196:3300/0 conn(0x7faa6c06eba0 0x7faa6c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667095607 0 0) 0x7faa6c06f110 con 0x7faa6c0ce190 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa58003530 con 0x7faa6c0ce190 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1520167479 0 0) 0x7faa58003530 con 0x7faa6c0ce190 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 >> v2:172.21.15.196:3300/0 conn(0x7faa6c06eba0 msgr2=0x7faa6c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 --2- 172.21.15.129:0/1921538028 >> v2:172.21.15.196:3300/0 conn(0x7faa6c06eba0 0x7faa6c15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 >> v1:172.21.15.196:6789/0 conn(0x7faa6c06cc40 legacy=0x7faa6c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c06f9b0 con 0x7faa6c0ce190 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7faa64001a00 con 0x7faa6c0ce190 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 >> v1:172.21.15.129:6789/0 conn(0x7faa6c0ce190 legacy=0x7faa6c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa580039c0 0x7faa58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa58004810 0x7faa58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa580051d0 0x7faa5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c06f6d0 con 0x7faa58004810 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa5800d180 con 0x7faa580051d0 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa5800c0c0 con 0x7faa580039c0 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa58004810 0x7faa58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6affd640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa580039c0 0x7faa58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6b7fe640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa580051d0 0x7faa5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 -- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa580039c0 msgr2=0x7faa58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa580039c0 0x7faa58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 -- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa580051d0 msgr2=0x7faa5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa580051d0 0x7faa5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa6b7fe640 1 -- 172.21.15.129:0/1921538028 reap_dead start 2024-04-25T00:42:52.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.480+0000 7faa63fff640 1 -- 172.21.15.129:0/1921538028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa58003530 con 0x7faa58004810 2024-04-25T00:42:52.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa63fff640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa58004810 0x7faa58004c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa54009ac0 tx=0x7faa5401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f4c303a68d89be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540b2070 con 0x7faa58004810 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa540a5530 con 0x7faa58004810 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa6a7fc640 1 -- 172.21.15.129:0/1921538028 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54009690 con 0x7faa58004810 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- 172.21.15.129:0/1921538028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa58004810 msgr2=0x7faa58004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa58004810 0x7faa58004c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa54009ac0 tx=0x7faa5401ca50 comp rx=0 tx=0).stop 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- 172.21.15.129:0/1921538028 shutdown_connections 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 --2- 172.21.15.129:0/1921538028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa58004810 0x7faa58004c90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- 172.21.15.129:0/1921538028 >> 172.21.15.129:0/1921538028 conn(0x7faa6c067730 msgr2=0x7faa6c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- 172.21.15.129:0/1921538028 shutdown_connections 2024-04-25T00:42:52.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- 172.21.15.129:0/1921538028 wait complete. 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 Processor -- start 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- start start 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa58004810 0x7faa6c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa6c13fe70 0x7faa6c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 0x7faa6c152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0aefd0 con 0x7faa6c1402f0 2024-04-25T00:42:52.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0cc410 con 0x7faa6c13fe70 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa713d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c140890 con 0x7faa58004810 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 0x7faa6c152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.484+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 0x7faa6c152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38938/0 (socket says 172.21.15.129:38938) 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 -- 172.21.15.129:0/383270578 learned_addr learned my addr 172.21.15.129:0/383270578 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa63fff640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa6c13fe70 0x7faa6c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6affd640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa58004810 0x7faa6c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 -- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa58004810 msgr2=0x7faa6c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa58004810 0x7faa6c156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 -- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa6c13fe70 msgr2=0x7faa6c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa6c13fe70 0x7faa6c159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 -- 172.21.15.129:0/383270578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c140b70 con 0x7faa6c1402f0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6b7fe640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 0x7faa6c152460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0ce570 tx=0x7faa64011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64010040 con 0x7faa6c1402f0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6401a070 con 0x7faa6c1402f0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640139c0 con 0x7faa6c1402f0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c140e50 con 0x7faa6c1402f0 2024-04-25T00:42:52.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c141410 con 0x7faa6c1402f0 2024-04-25T00:42:52.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c062e50 con 0x7faa6c1402f0 2024-04-25T00:42:52.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa64013b60 con 0x7faa6c1402f0 2024-04-25T00:42:52.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa637fe640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa4c0678f0 0x7faa4c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7faa64088710 con 0x7faa6c1402f0 2024-04-25T00:42:52.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6affd640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa4c0678f0 0x7faa4c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.488+0000 7faa6affd640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa4c0678f0 0x7faa4c069db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa6c158f10 tx=0x7faa5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.492+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faa640595a0 con 0x7faa6c1402f0 2024-04-25T00:42:52.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa6c075320 con 0x7faa6c1402f0 2024-04-25T00:42:52.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa637fe640 1 -- 172.21.15.129:0/383270578 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7faa64058ea0 con 0x7faa6c1402f0 2024-04-25T00:42:52.647 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa4c0678f0 msgr2=0x7faa4c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa4c0678f0 0x7faa4c069db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa6c158f10 tx=0x7faa5c009040 comp rx=0 tx=0).stop 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 msgr2=0x7faa6c152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 0x7faa6c152460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0ce570 tx=0x7faa64011a10 comp rx=0 tx=0).stop 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 shutdown_connections 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa4c0678f0 0x7faa4c069db0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa6c1402f0 0x7faa6c152460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa6c13fe70 0x7faa6c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 --2- 172.21.15.129:0/383270578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa58004810 0x7faa6c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 >> 172.21.15.129:0/383270578 conn(0x7faa6c067730 msgr2=0x7faa6c070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 shutdown_connections 2024-04-25T00:42:52.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.644+0000 7faa713d4640 1 -- 172.21.15.129:0/383270578 wait complete. 2024-04-25T00:42:52.664 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:42:52.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.764+0000 7fe1d9866640 1 Processor -- start 2024-04-25T00:42:52.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.764+0000 7fe1d9866640 1 -- start start 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d9866640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1d415e240 0x7fe1d406d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d9866640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe1d40ae250 0x7fe1d40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d9866640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d40b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d9866640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d40ceb10 con 0x7fe1d415e240 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d9866640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d40adc50 con 0x7fe1d40ae250 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d9866640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d40ced80 con 0x7fe1d40ae6b0 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1d415e240 0x7fe1d406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1d415e240 0x7fe1d406d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38944/0 (socket says 172.21.15.129:38944) 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 -- 172.21.15.129:0/2618671335 learned_addr learned my addr 172.21.15.129:0/2618671335 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d37fe640 1 --2- 172.21.15.129:0/2618671335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d40b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1cbfff640 1 --2- 172.21.15.129:0/2618671335 >> v2:172.21.15.196:3300/0 conn(0x7fe1d40ae250 0x7fe1d40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 -- 172.21.15.129:0/2618671335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 msgr2=0x7fe1d40b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/2618671335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d40b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 -- 172.21.15.129:0/2618671335 >> v2:172.21.15.196:3300/0 conn(0x7fe1d40ae250 msgr2=0x7fe1d40b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:52.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/2618671335 >> v2:172.21.15.196:3300/0 conn(0x7fe1d40ae250 0x7fe1d40b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 -- 172.21.15.129:0/2618671335 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d40cf060 con 0x7fe1d415e240 2024-04-25T00:42:52.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/2618671335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d406d730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40b40f0 tx=0x7fe1c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab430676cde653e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d27fc640 1 -- 172.21.15.129:0/2618671335 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40b0070 con 0x7fe1d415e240 2024-04-25T00:42:52.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d27fc640 1 -- 172.21.15.129:0/2618671335 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c40ab600 con 0x7fe1d415e240 2024-04-25T00:42:52.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.768+0000 7fe1d27fc640 1 -- 172.21.15.129:0/2618671335 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40ab9d0 con 0x7fe1d415e240 2024-04-25T00:42:52.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/2618671335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 msgr2=0x7fe1d406d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- 172.21.15.129:0/2618671335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d406d730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40b40f0 tx=0x7fe1c401c520 comp rx=0 tx=0).stop 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/2618671335 shutdown_connections 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- 172.21.15.129:0/2618671335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d40b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- 172.21.15.129:0/2618671335 >> v2:172.21.15.196:3300/0 conn(0x7fe1d40ae250 0x7fe1d40b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- 172.21.15.129:0/2618671335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d406d730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/2618671335 >> 172.21.15.129:0/2618671335 conn(0x7fe1d40b3190 msgr2=0x7fe1d40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/2618671335 shutdown_connections 2024-04-25T00:42:52.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/2618671335 wait complete. 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 Processor -- start 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- start start 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1d40ae250 0x7fe1d415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d4147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d4143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d40cff10 con 0x7fe1d415e240 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d40cfd90 con 0x7fe1d40ae250 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d40d0090 con 0x7fe1d40ae6b0 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d4143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d4143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38958/0 (socket says 172.21.15.129:38958) 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 -- 172.21.15.129:0/1060672772 learned_addr learned my addr 172.21.15.129:0/1060672772 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1cbfff640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d4147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1d40ae250 0x7fe1d415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 -- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 msgr2=0x7fe1d4147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d4147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 -- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1d40ae250 msgr2=0x7fe1d415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1d40ae250 0x7fe1d415d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 -- 172.21.15.129:0/1060672772 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d415bfd0 con 0x7fe1d415e240 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1d40ae250 0x7fe1d415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:52.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d37fe640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d4143a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc007de0 tx=0x7fe1cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc014620 con 0x7fe1d415e240 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc019070 con 0x7fe1d415e240 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d415c270 con 0x7fe1d415e240 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc014a20 con 0x7fe1d415e240 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.772+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d4143f90 con 0x7fe1d415e240 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.776+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d4062e50 con 0x7fe1d415e240 2024-04-25T00:42:52.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.776+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1cc014bc0 con 0x7fe1d415e240 2024-04-25T00:42:52.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.776+0000 7fe1d0ff9640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1b40678f0 0x7fe1b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:52.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.776+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1b40678f0 0x7fe1b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:52.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.776+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe1cc086260 con 0x7fe1d415e240 2024-04-25T00:42:52.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.776+0000 7fe1d2ffd640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1b40678f0 0x7fe1b4069db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40ad000 tx=0x7fe1c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:52.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.780+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe1cc0570f0 con 0x7fe1d415e240 2024-04-25T00:42:52.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.976+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1d4067750 con 0x7fe1d415e240 2024-04-25T00:42:52.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.976+0000 7fe1d0ff9640 1 -- 172.21.15.129:0/1060672772 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fe1cc0569f0 con 0x7fe1d415e240 2024-04-25T00:42:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:52.982 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:42:52.982 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 2 2024-04-25T00:42:52.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1b40678f0 msgr2=0x7fe1b4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1b40678f0 0x7fe1b4069db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40ad000 tx=0x7fe1c4008040 comp rx=0 tx=0).stop 2024-04-25T00:42:52.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 msgr2=0x7fe1d4143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d4143a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc007de0 tx=0x7fe1cc00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 shutdown_connections 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1b40678f0 0x7fe1b4069db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1d40ae6b0 0x7fe1d4147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1d415e240 0x7fe1d4143a50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 --2- 172.21.15.129:0/1060672772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1d40ae250 0x7fe1d415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 >> 172.21.15.129:0/1060672772 conn(0x7fe1d40b3190 msgr2=0x7fe1d40cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 shutdown_connections 2024-04-25T00:42:52.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:52.980+0000 7fe1d9866640 1 -- 172.21.15.129:0/1060672772 wait complete. 2024-04-25T00:42:55.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:42:55.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 Processor -- start 2024-04-25T00:42:55.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 -- start start 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff59415c7f0 0x7ff5940723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff5940cd190 0x7ff59415c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff594072aa0 con 0x7ff5940ccdb0 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff594070690 con 0x7ff5940cd190 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff59b9f6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff5940708e0 con 0x7ff59415c7f0 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff5940cd190 0x7ff59415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff5940cd190 0x7ff59415c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52704/0 (socket says 172.21.15.129:52704) 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599792640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff5940ccdb0 0x7ff594158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60096/0 (socket says 172.21.15.129:60096) 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 -- 172.21.15.129:0/1754443234 learned_addr learned my addr 172.21.15.129:0/1754443234 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:55.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff598f91640 1 --2- 172.21.15.129:0/1754443234 >> v2:172.21.15.196:3300/0 conn(0x7ff59415c7f0 0x7ff5940723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 -- 172.21.15.129:0/1754443234 >> v2:172.21.15.196:3300/0 conn(0x7ff59415c7f0 msgr2=0x7ff5940723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:55.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 --2- 172.21.15.129:0/1754443234 >> v2:172.21.15.196:3300/0 conn(0x7ff59415c7f0 0x7ff5940723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 -- 172.21.15.129:0/1754443234 >> v1:172.21.15.129:6789/0 conn(0x7ff5940ccdb0 legacy=0x7ff594158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff58bfff640 1 -- 172.21.15.129:0/1754443234 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2480736651 0 0) 0x7ff594072aa0 con 0x7ff5940ccdb0 2024-04-25T00:42:55.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff599f93640 1 -- 172.21.15.129:0/1754443234 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff594070bc0 con 0x7ff5940cd190 2024-04-25T00:42:55.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.128+0000 7ff598f91640 1 --2- 172.21.15.129:0/1754443234 >> v2:172.21.15.196:3300/0 conn(0x7ff59415c7f0 0x7ff5940723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff599f93640 1 --2- 172.21.15.129:0/1754443234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59415c120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff594072d80 tx=0x7ff58c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63116e468e7d38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff58bfff640 1 -- 172.21.15.129:0/1754443234 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0b0070 con 0x7ff5940cd190 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff58bfff640 1 -- 172.21.15.129:0/1754443234 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c0ab600 con 0x7ff5940cd190 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff58bfff640 1 -- 172.21.15.129:0/1754443234 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0ab9d0 con 0x7ff5940cd190 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1754443234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 msgr2=0x7ff59415c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1754443234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59415c120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff594072d80 tx=0x7ff58c01c490 comp rx=0 tx=0).stop 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1754443234 shutdown_connections 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1754443234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59415c120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1754443234 >> v2:172.21.15.196:3300/0 conn(0x7ff59415c7f0 0x7ff5940723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1754443234 >> 172.21.15.129:0/1754443234 conn(0x7ff5940b14a0 msgr2=0x7ff5940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1754443234 shutdown_connections 2024-04-25T00:42:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1754443234 wait complete. 2024-04-25T00:42:55.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 Processor -- start 2024-04-25T00:42:55.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- start start 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5940ccdb0 0x7ff594071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59413caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff59415c7f0 0x7ff5940b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff59406ea10 con 0x7ff5940cd190 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff59406e890 con 0x7ff5940ccdb0 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff59b9f6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff59406eb90 con 0x7ff59415c7f0 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff598f91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59413caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff598f91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59413caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52706/0 (socket says 172.21.15.129:52706) 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.132+0000 7ff598f91640 1 -- 172.21.15.129:0/1949834913 learned_addr learned my addr 172.21.15.129:0/1949834913 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff599792640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5940ccdb0 0x7ff594071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff599f93640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff59415c7f0 0x7ff5940b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff598f91640 1 -- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff59415c7f0 msgr2=0x7ff5940b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff598f91640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff59415c7f0 0x7ff5940b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff598f91640 1 -- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5940ccdb0 msgr2=0x7ff594071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff598f91640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5940ccdb0 0x7ff594071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff598f91640 1 -- 172.21.15.129:0/1949834913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff594155a80 con 0x7ff5940cd190 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff599792640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5940ccdb0 0x7ff594071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff598f91640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59413caf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c820 tx=0x7ff58400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584015070 con 0x7ff5940cd190 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584007a30 con 0x7ff5940cd190 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff594155d60 con 0x7ff5940cd190 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584007e40 con 0x7ff5940cd190 2024-04-25T00:42:55.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff594156320 con 0x7ff5940cd190 2024-04-25T00:42:55.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff594062e50 con 0x7ff5940cd190 2024-04-25T00:42:55.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff584018070 con 0x7ff5940cd190 2024-04-25T00:42:55.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff589ffb640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff5780678c0 0x7ff578069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff584085910 con 0x7ff5940cd190 2024-04-25T00:42:55.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff599792640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff5780678c0 0x7ff578069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.136+0000 7ff599792640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff5780678c0 0x7ff578069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5900012f0 tx=0x7ff590009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.140+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff584056710 con 0x7ff5940cd190 2024-04-25T00:42:55.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.320+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff594075320 con 0x7ff5940cd190 2024-04-25T00:42:55.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.320+0000 7ff589ffb640 1 -- 172.21.15.129:0/1949834913 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7ff584056010 con 0x7ff5940cd190 2024-04-25T00:42:55.325 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:55.325 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:42:55.325 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 3 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff5780678c0 msgr2=0x7ff578069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff5780678c0 0x7ff578069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5900012f0 tx=0x7ff590009040 comp rx=0 tx=0).stop 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 msgr2=0x7ff59413caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59413caf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c820 tx=0x7ff58400cce0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 shutdown_connections 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff5780678c0 0x7ff578069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff59415c7f0 0x7ff5940b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5940cd190 0x7ff59413caf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 --2- 172.21.15.129:0/1949834913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5940ccdb0 0x7ff594071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 >> 172.21.15.129:0/1949834913 conn(0x7ff5940b14a0 msgr2=0x7ff594152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:55.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 shutdown_connections 2024-04-25T00:42:55.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.324+0000 7ff59b9f6640 1 -- 172.21.15.129:0/1949834913 wait complete. 2024-04-25T00:42:55.343 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:55.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.456+0000 7f60de82d640 1 Processor -- start 2024-04-25T00:42:55.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.456+0000 7f60de82d640 1 -- start start 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60de82d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f60d80cd5c0 0x7f60d806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60de82d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f60d81417d0 0x7f60d81603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60de82d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d8141c50 con 0x7f60d81608f0 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60de82d640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f60d813f490 con 0x7f60d81417d0 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60de82d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f60d8141f00 con 0x7f60d80cd5c0 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d77fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f60d81417d0 0x7f60d81603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60dcdca640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f60d81608f0 0x7f60d815c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60102/0 (socket says 172.21.15.129:60102) 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60dcdca640 1 -- 172.21.15.129:0/4018066853 learned_addr learned my addr 172.21.15.129:0/4018066853 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d7fff640 1 --2- 172.21.15.129:0/4018066853 >> v2:172.21.15.196:3300/0 conn(0x7f60d80cd5c0 0x7f60d806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d77fe640 1 -- 172.21.15.129:0/4018066853 >> v2:172.21.15.196:3300/0 conn(0x7f60d80cd5c0 msgr2=0x7f60d806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d6ffd640 1 -- 172.21.15.129:0/4018066853 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758584525 0 0) 0x7f60d8141c50 con 0x7f60d81608f0 2024-04-25T00:42:55.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d77fe640 1 --2- 172.21.15.129:0/4018066853 >> v2:172.21.15.196:3300/0 conn(0x7f60d80cd5c0 0x7f60d806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d77fe640 1 -- 172.21.15.129:0/4018066853 >> v1:172.21.15.129:6789/0 conn(0x7f60d81608f0 legacy=0x7f60d815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d77fe640 1 -- 172.21.15.129:0/4018066853 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d81421e0 con 0x7f60d81417d0 2024-04-25T00:42:55.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d7fff640 1 --2- 172.21.15.129:0/4018066853 >> v2:172.21.15.196:3300/0 conn(0x7f60d80cd5c0 0x7f60d806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:55.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d77fe640 1 --2- 172.21.15.129:0/4018066853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81417d0 0x7f60d81603b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60d806c2d0 tx=0x7f60cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23b5dd806320029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d6ffd640 1 -- 172.21.15.129:0/4018066853 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b0070 con 0x7f60d81417d0 2024-04-25T00:42:55.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d6ffd640 1 -- 172.21.15.129:0/4018066853 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc0ab600 con 0x7f60d81417d0 2024-04-25T00:42:55.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.460+0000 7f60d6ffd640 1 -- 172.21.15.129:0/4018066853 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0ab9d0 con 0x7f60d81417d0 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/4018066853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81417d0 msgr2=0x7f60d81603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 --2- 172.21.15.129:0/4018066853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81417d0 0x7f60d81603b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60d806c2d0 tx=0x7f60cc01c480 comp rx=0 tx=0).stop 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/4018066853 shutdown_connections 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 --2- 172.21.15.129:0/4018066853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81417d0 0x7f60d81603b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 --2- 172.21.15.129:0/4018066853 >> v2:172.21.15.196:3300/0 conn(0x7f60d80cd5c0 0x7f60d806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/4018066853 >> 172.21.15.129:0/4018066853 conn(0x7f60d80b2380 msgr2=0x7f60d80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/4018066853 shutdown_connections 2024-04-25T00:42:55.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/4018066853 wait complete. 2024-04-25T00:42:55.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 Processor -- start 2024-04-25T00:42:55.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- start start 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d80cd5c0 0x7f60d813a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60d81417d0 0x7f60d8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 0x7f60d8153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60d80734e0 con 0x7f60d81608f0 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8073360 con 0x7f60d80cd5c0 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60d8070970 con 0x7f60d81417d0 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 0x7f60d8153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 0x7f60d8153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52718/0 (socket says 172.21.15.129:52718) 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 -- 172.21.15.129:0/1611879043 learned_addr learned my addr 172.21.15.129:0/1611879043 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60d7fff640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d80cd5c0 0x7f60d813a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60d77fe640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60d81417d0 0x7f60d8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 -- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60d81417d0 msgr2=0x7f60d8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60d81417d0 0x7f60d8154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 -- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d80cd5c0 msgr2=0x7f60d813a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d80cd5c0 0x7f60d813a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 -- 172.21.15.129:0/1611879043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d8153ad0 con 0x7f60d81608f0 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60dcdca640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 0x7f60d8153050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60d000c700 tx=0x7f60d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0018070 con 0x7f60d81608f0 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d000ce10 con 0x7f60d81608f0 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d8153db0 con 0x7f60d81608f0 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000a800 con 0x7f60d81608f0 2024-04-25T00:42:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.464+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d8154370 con 0x7f60d81608f0 2024-04-25T00:42:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.468+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d8062e50 con 0x7f60d81608f0 2024-04-25T00:42:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.468+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60d001b070 con 0x7f60d81608f0 2024-04-25T00:42:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.468+0000 7f60d4ff9640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f60c40678c0 0x7f60c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.468+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f60d0084db0 con 0x7f60d81608f0 2024-04-25T00:42:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.468+0000 7f60d7fff640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f60c40678c0 0x7f60c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.468+0000 7f60d7fff640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f60c40678c0 0x7f60c4069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f60c8006fd0 tx=0x7f60c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.472+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f60d0055cd0 con 0x7f60d81608f0 2024-04-25T00:42:55.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.628+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60d8075320 con 0x7f60d81608f0 2024-04-25T00:42:55.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.628+0000 7f60d4ff9640 1 -- 172.21.15.129:0/1611879043 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f60d00555d0 con 0x7f60d81608f0 2024-04-25T00:42:55.633 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:55.633 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":16,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:48.886021+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f60c40678c0 msgr2=0x7f60c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f60c40678c0 0x7f60c4069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f60c8006fd0 tx=0x7f60c8008040 comp rx=0 tx=0).stop 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 msgr2=0x7f60d8153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 0x7f60d8153050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60d000c700 tx=0x7f60d000cbc0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 shutdown_connections 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f60c40678c0 0x7f60c4069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60d81417d0 0x7f60d8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60d81608f0 0x7f60d8153050 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 --2- 172.21.15.129:0/1611879043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d80cd5c0 0x7f60d813a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 >> 172.21.15.129:0/1611879043 conn(0x7f60d80b2380 msgr2=0x7f60d815c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:55.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 shutdown_connections 2024-04-25T00:42:55.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.632+0000 7f60de82d640 1 -- 172.21.15.129:0/1611879043 wait complete. 2024-04-25T00:42:55.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:42:55.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:42:55.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.772+0000 7f0763ddb640 1 Processor -- start 2024-04-25T00:42:55.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.772+0000 7f0763ddb640 1 -- start start 2024-04-25T00:42:55.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0763ddb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f075c145960 0x7f075c160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0763ddb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c15c590 0x7f075c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0763ddb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075c06dbf0 con 0x7f075c0cccd0 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0763ddb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f075c0cd140 con 0x7f075c145960 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0763ddb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f075c160ad0 con 0x7f075c15c590 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761376640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c15c590 0x7f075c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761b77640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f075c145960 0x7f075c160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761376640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c15c590 0x7f075c0cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:48154/0 (socket says 172.21.15.129:48154) 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761376640 1 -- 172.21.15.129:0/1705414159 learned_addr learned my addr 172.21.15.129:0/1705414159 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0762378640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f075c0cccd0 0x7f075c0cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:59778/0 (socket says 172.21.15.129:59778) 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761b77640 1 -- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c15c590 msgr2=0x7f075c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761b77640 1 --2- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c15c590 0x7f075c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761b77640 1 -- 172.21.15.129:0/1705414159 >> v1:172.21.15.196:6789/0 conn(0x7f075c0cccd0 legacy=0x7f075c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.776+0000 7f0761b77640 1 -- 172.21.15.129:0/1705414159 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c06f090 con 0x7f075c145960 2024-04-25T00:42:55.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0761b77640 1 --2- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c145960 0x7f075c160590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f07580097c0 tx=0x7f075801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bdcc9336902491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0760b75640 1 -- 172.21.15.129:0/1705414159 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07580b0070 con 0x7f075c145960 2024-04-25T00:42:55.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0760b75640 1 -- 172.21.15.129:0/1705414159 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07580ab600 con 0x7f075c145960 2024-04-25T00:42:55.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0760b75640 1 -- 172.21.15.129:0/1705414159 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07580ab9d0 con 0x7f075c145960 2024-04-25T00:42:55.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c145960 msgr2=0x7f075c160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c145960 0x7f075c160590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f07580097c0 tx=0x7f075801c3e0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- 172.21.15.129:0/1705414159 shutdown_connections 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c15c590 0x7f075c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1705414159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c145960 0x7f075c160590 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- 172.21.15.129:0/1705414159 >> 172.21.15.129:0/1705414159 conn(0x7f075c067730 msgr2=0x7f075c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- 172.21.15.129:0/1705414159 shutdown_connections 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- 172.21.15.129:0/1705414159 wait complete. 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 Processor -- start 2024-04-25T00:42:55.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- start start 2024-04-25T00:42:55.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c0cccd0 0x7f075c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c145960 0x7f075c1650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 0x7f075c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0cf890 con 0x7f075c15c590 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f075c06d880 con 0x7f075c0cccd0 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f075c06cc40 con 0x7f075c145960 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 0x7f075c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 0x7f075c13f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52724/0 (socket says 172.21.15.129:52724) 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 -- 172.21.15.129:0/1092408007 learned_addr learned my addr 172.21.15.129:0/1092408007 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0761376640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c145960 0x7f075c1650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0761b77640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c0cccd0 0x7f075c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 -- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c145960 msgr2=0x7f075c1650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c145960 0x7f075c1650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 -- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c0cccd0 msgr2=0x7f075c1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c0cccd0 0x7f075c1405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 -- 172.21.15.129:0/1092408007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c165f80 con 0x7f075c15c590 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0762378640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 0x7f075c13f6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f075400c7d0 tx=0x7f075400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754017070 con 0x7f075c15c590 2024-04-25T00:42:55.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075400a640 con 0x7f075c15c590 2024-04-25T00:42:55.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.780+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1659c0 con 0x7f075c15c590 2024-04-25T00:42:55.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075400a9f0 con 0x7f075c15c590 2024-04-25T00:42:55.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c070730 con 0x7f075c15c590 2024-04-25T00:42:55.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c062e50 con 0x7f075c15c590 2024-04-25T00:42:55.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f075401a070 con 0x7f075c15c590 2024-04-25T00:42:55.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f07527fc640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f07400678c0 0x7f0740069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:55.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0754084e40 con 0x7f075c15c590 2024-04-25T00:42:55.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f0761b77640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f07400678c0 0x7f0740069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:55.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.784+0000 7f0761b77640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f07400678c0 0x7f0740069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07580ad000 tx=0x7f0758008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:55.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.788+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0754055710 con 0x7f075c15c590 2024-04-25T00:42:55.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.948+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f075c075320 con 0x7f075c15c590 2024-04-25T00:42:55.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.948+0000 7f07527fc640 1 -- 172.21.15.129:0/1092408007 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f0754059a10 con 0x7f075c15c590 2024-04-25T00:42:55.954 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f07400678c0 msgr2=0x7f0740069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f07400678c0 0x7f0740069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07580ad000 tx=0x7f0758008040 comp rx=0 tx=0).stop 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 msgr2=0x7f075c13f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 0x7f075c13f6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f075400c7d0 tx=0x7f075400cc90 comp rx=0 tx=0).stop 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 shutdown_connections 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f07400678c0 0x7f0740069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f075c145960 0x7f075c1650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f075c15c590 0x7f075c13f6a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 --2- 172.21.15.129:0/1092408007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f075c0cccd0 0x7f075c1405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 >> 172.21.15.129:0/1092408007 conn(0x7f075c067730 msgr2=0x7f075c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 shutdown_connections 2024-04-25T00:42:55.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:55.952+0000 7f0763ddb640 1 -- 172.21.15.129:0/1092408007 wait complete. 2024-04-25T00:42:55.973 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:42:56.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.080+0000 7fe26e922640 1 Processor -- start 2024-04-25T00:42:56.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.080+0000 7fe26e922640 1 -- start start 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26e922640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe26815bf50 0x7fe26815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26e922640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2680aa0c0 0x7fe2680aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26e922640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26806d1b0 con 0x7fe2680ccb50 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26e922640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fe26806d490 con 0x7fe26815bf50 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26e922640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe26806d770 con 0x7fe2680aa0c0 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe267fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe26815bf50 0x7fe26815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26cebf640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fe2680ccb50 0x7fe26806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60106/0 (socket says 172.21.15.129:60106) 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe26cebf640 1 -- 172.21.15.129:0/1047595624 learned_addr learned my addr 172.21.15.129:0/1047595624 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe2677fe640 1 --2- 172.21.15.129:0/1047595624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2680aa0c0 0x7fe2680aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe267fff640 1 -- 172.21.15.129:0/1047595624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2680aa0c0 msgr2=0x7fe2680aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe267fff640 1 --2- 172.21.15.129:0/1047595624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2680aa0c0 0x7fe2680aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe267fff640 1 -- 172.21.15.129:0/1047595624 >> v1:172.21.15.129:6789/0 conn(0x7fe2680ccb50 legacy=0x7fe26806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe267fff640 1 -- 172.21.15.129:0/1047595624 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26806da50 con 0x7fe26815bf50 2024-04-25T00:42:56.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe267fff640 1 --2- 172.21.15.129:0/1047595624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe26815bf50 0x7fe26815c330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2580097c0 tx=0x7fe25801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce06a4d6f4731d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe266ffd640 1 -- 172.21.15.129:0/1047595624 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580b0070 con 0x7fe26815bf50 2024-04-25T00:42:56.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe266ffd640 1 -- 172.21.15.129:0/1047595624 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2580ab600 con 0x7fe26815bf50 2024-04-25T00:42:56.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.084+0000 7fe266ffd640 1 -- 172.21.15.129:0/1047595624 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580ab9d0 con 0x7fe26815bf50 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/1047595624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe26815bf50 msgr2=0x7fe26815c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 --2- 172.21.15.129:0/1047595624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe26815bf50 0x7fe26815c330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2580097c0 tx=0x7fe25801c330 comp rx=0 tx=0).stop 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/1047595624 shutdown_connections 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 --2- 172.21.15.129:0/1047595624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2680aa0c0 0x7fe2680aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 --2- 172.21.15.129:0/1047595624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe26815bf50 0x7fe26815c330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/1047595624 >> 172.21.15.129:0/1047595624 conn(0x7fe2680b14a0 msgr2=0x7fe2680b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/1047595624 shutdown_connections 2024-04-25T00:42:56.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/1047595624 wait complete. 2024-04-25T00:42:56.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 Processor -- start 2024-04-25T00:42:56.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- start start 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2680aa0c0 0x7fe2680b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 0x7fe268146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26815bf50 0x7fe268156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe26815c9f0 con 0x7fe2680ccb50 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe26815c870 con 0x7fe2680aa0c0 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe26806cd60 con 0x7fe26815bf50 2024-04-25T00:42:56.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 0x7fe268146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 0x7fe268146b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52732/0 (socket says 172.21.15.129:52732) 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 -- 172.21.15.129:0/386026712 learned_addr learned my addr 172.21.15.129:0/386026712 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26cebf640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26815bf50 0x7fe268156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe267fff640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2680aa0c0 0x7fe2680b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 -- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26815bf50 msgr2=0x7fe268156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26815bf50 0x7fe268156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 -- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2680aa0c0 msgr2=0x7fe2680b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2680aa0c0 0x7fe2680b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 -- 172.21.15.129:0/386026712 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe268157220 con 0x7fe2680ccb50 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe2677fe640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 0x7fe268146b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00d5f0 tx=0x7fe25c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0125f0 con 0x7fe2680ccb50 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25c019070 con 0x7fe2680ccb50 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2681574f0 con 0x7fe2680ccb50 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0129d0 con 0x7fe2680ccb50 2024-04-25T00:42:56.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.088+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe268152ca0 con 0x7fe2680ccb50 2024-04-25T00:42:56.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.092+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe268062e50 con 0x7fe2680ccb50 2024-04-25T00:42:56.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.092+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe25c012b70 con 0x7fe2680ccb50 2024-04-25T00:42:56.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.092+0000 7fe264ff9640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe2540678c0 0x7fe254069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.092+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe25c085dd0 con 0x7fe2680ccb50 2024-04-25T00:42:56.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.092+0000 7fe267fff640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe2540678c0 0x7fe254069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.092+0000 7fe267fff640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe2540678c0 0x7fe254069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2580ad000 tx=0x7fe2580a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.096+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe25c056c60 con 0x7fe2680ccb50 2024-04-25T00:42:56.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.256+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe268075320 con 0x7fe2680ccb50 2024-04-25T00:42:56.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.256+0000 7fe264ff9640 1 -- 172.21.15.129:0/386026712 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fe25c087090 con 0x7fe2680ccb50 2024-04-25T00:42:56.262 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:56.262 INFO:teuthology.orchestra.run.smithi129.stdout:[ 2024-04-25T00:42:56.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-25T00:42:56.262 INFO:teuthology.orchestra.run.smithi129.stdout: "poolnum": 1, 2024-04-25T00:42:56.262 INFO:teuthology.orchestra.run.smithi129.stdout: "poolname": ".mgr" 2024-04-25T00:42:56.263 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T00:42:56.263 INFO:teuthology.orchestra.run.smithi129.stdout:] 2024-04-25T00:42:56.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe2540678c0 msgr2=0x7fe254069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe2540678c0 0x7fe254069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2580ad000 tx=0x7fe2580a6000 comp rx=0 tx=0).stop 2024-04-25T00:42:56.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 msgr2=0x7fe268146b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 0x7fe268146b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00d5f0 tx=0x7fe25c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 shutdown_connections 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe2540678c0 0x7fe254069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26815bf50 0x7fe268156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2680ccb50 0x7fe268146b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 --2- 172.21.15.129:0/386026712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2680aa0c0 0x7fe2680b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 >> 172.21.15.129:0/386026712 conn(0x7fe2680b14a0 msgr2=0x7fe268158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 shutdown_connections 2024-04-25T00:42:56.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.260+0000 7fe26e922640 1 -- 172.21.15.129:0/386026712 wait complete. 2024-04-25T00:42:56.281 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:42:56.281 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:42:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 Processor -- start 2024-04-25T00:42:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 -- start start 2024-04-25T00:42:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8664144fc0 0x7f86640ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f866406f560 0x7f86640ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86640ced50 con 0x7f86641608f0 2024-04-25T00:42:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86641453a0 con 0x7f866406f560 2024-04-25T00:42:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866c535640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f866415fbc0 con 0x7f8664144fc0 2024-04-25T00:42:56.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866aad2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f866406f560 0x7f86640ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f8669ad0640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f86641608f0 0x7f866406ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:59790/0 (socket says 172.21.15.129:59790) 2024-04-25T00:42:56.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f8669ad0640 1 -- 172.21.15.129:0/1389535324 learned_addr learned my addr 172.21.15.129:0/1389535324 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:56.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866a2d1640 1 --2- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8664144fc0 0x7f86640ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f86692cf640 1 -- 172.21.15.129:0/1389535324 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069336773 0 0) 0x7f86640ced50 con 0x7f86641608f0 2024-04-25T00:42:56.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f86692cf640 1 -- 172.21.15.129:0/1389535324 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8650003530 con 0x7f86641608f0 2024-04-25T00:42:56.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866aad2640 1 -- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8664144fc0 msgr2=0x7f86640ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:56.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866aad2640 1 --2- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8664144fc0 0x7f86640ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866aad2640 1 -- 172.21.15.129:0/1389535324 >> v1:172.21.15.196:6789/0 conn(0x7f86641608f0 legacy=0x7f866406ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.400+0000 7f866aad2640 1 -- 172.21.15.129:0/1389535324 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866415fe10 con 0x7f866406f560 2024-04-25T00:42:56.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866aad2640 1 --2- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f866406f560 0x7f86640ccac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f865c009d90 tx=0x7f865c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad5b22d5bf69d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f86692cf640 1 -- 172.21.15.129:0/1389535324 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b2070 con 0x7f866406f560 2024-04-25T00:42:56.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f86692cf640 1 -- 172.21.15.129:0/1389535324 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c0a5970 con 0x7f866406f560 2024-04-25T00:42:56.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f86692cf640 1 -- 172.21.15.129:0/1389535324 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0a5d40 con 0x7f866406f560 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 -- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f866406f560 msgr2=0x7f86640ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 --2- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f866406f560 0x7f86640ccac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f865c009d90 tx=0x7f865c01c8b0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 -- 172.21.15.129:0/1389535324 shutdown_connections 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 --2- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f866406f560 0x7f86640ccac0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 --2- 172.21.15.129:0/1389535324 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8664144fc0 0x7f86640ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 -- 172.21.15.129:0/1389535324 >> 172.21.15.129:0/1389535324 conn(0x7f8664067760 msgr2=0x7f86640b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 -- 172.21.15.129:0/1389535324 shutdown_connections 2024-04-25T00:42:56.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 -- 172.21.15.129:0/1389535324 wait complete. 2024-04-25T00:42:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.404+0000 7f866c535640 1 Processor -- start 2024-04-25T00:42:56.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- start start 2024-04-25T00:42:56.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f866406f560 0x7f866415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8664144fc0 0x7f86640b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 0x7f866413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86641690e0 con 0x7f86641608f0 2024-04-25T00:42:56.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8664168f60 con 0x7f8664144fc0 2024-04-25T00:42:56.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8664169260 con 0x7f866406f560 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 0x7f866413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 0x7f866413ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52738/0 (socket says 172.21.15.129:52738) 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 -- 172.21.15.129:0/889859843 learned_addr learned my addr 172.21.15.129:0/889859843 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866a2d1640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f866406f560 0x7f866415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f8669ad0640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8664144fc0 0x7f86640b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 -- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f866406f560 msgr2=0x7f866415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f866406f560 0x7f866415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 -- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8664144fc0 msgr2=0x7f86640b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8664144fc0 0x7f86640b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 -- 172.21.15.129:0/889859843 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866413a140 con 0x7f86641608f0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866aad2640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 0x7f866413ff10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f865c009d90 tx=0x7f865c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0bb020 con 0x7f86641608f0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865c0ad440 con 0x7f86641608f0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866413a420 con 0x7f86641608f0 2024-04-25T00:42:56.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b2040 con 0x7f86641608f0 2024-04-25T00:42:56.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866413a9e0 con 0x7f86641608f0 2024-04-25T00:42:56.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8664062e50 con 0x7f86641608f0 2024-04-25T00:42:56.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f865c0be020 con 0x7f86641608f0 2024-04-25T00:42:56.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f865affd640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f86480678c0 0x7f8648069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f865c11fe80 con 0x7f86641608f0 2024-04-25T00:42:56.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.408+0000 7f866a2d1640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f86480678c0 0x7f8648069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.412+0000 7f866a2d1640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f86480678c0 0x7f8648069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f866415f700 tx=0x7f8660008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.416+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f865c016560 con 0x7f86641608f0 2024-04-25T00:42:56.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.568+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8664075320 con 0x7f86641608f0 2024-04-25T00:42:56.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.620+0000 7f865affd640 1 -- 172.21.15.129:0/889859843 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f865c0f0d10 con 0x7f86641608f0 2024-04-25T00:42:56.623 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' created 2024-04-25T00:42:56.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f86480678c0 msgr2=0x7f8648069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f86480678c0 0x7f8648069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f866415f700 tx=0x7f8660008040 comp rx=0 tx=0).stop 2024-04-25T00:42:56.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 msgr2=0x7f866413ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 0x7f866413ff10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f865c009d90 tx=0x7f865c019650 comp rx=0 tx=0).stop 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 shutdown_connections 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f86480678c0 0x7f8648069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86641608f0 0x7f866413ff10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8664144fc0 0x7f86640b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 --2- 172.21.15.129:0/889859843 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f866406f560 0x7f866415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 >> 172.21.15.129:0/889859843 conn(0x7f8664067760 msgr2=0x7f86640af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:56.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 shutdown_connections 2024-04-25T00:42:56.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.624+0000 7f866c535640 1 -- 172.21.15.129:0/889859843 wait complete. 2024-04-25T00:42:56.648 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:42:56.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.768+0000 7fd37ac84640 1 Processor -- start 2024-04-25T00:42:56.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.768+0000 7fd37ac84640 1 -- start start 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd37415e180 0x7fd3740ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd374162660 0x7fd3740706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3740ce6a0 con 0x7fd374070c10 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd3740cf220 con 0x7fd374162660 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd3740ce980 con 0x7fd37415e180 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd374162660 0x7fd3740706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd374162660 0x7fd3740706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52744/0 (socket says 172.21.15.129:52744) 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd379221640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fd374070c10 0x7fd37406dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60112/0 (socket says 172.21.15.129:60112) 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 -- 172.21.15.129:0/2871777880 learned_addr learned my addr 172.21.15.129:0/2871777880 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd378a20640 1 --2- 172.21.15.129:0/2871777880 >> v2:172.21.15.196:3300/0 conn(0x7fd37415e180 0x7fd3740ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 -- 172.21.15.129:0/2871777880 >> v2:172.21.15.196:3300/0 conn(0x7fd37415e180 msgr2=0x7fd3740ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 --2- 172.21.15.129:0/2871777880 >> v2:172.21.15.196:3300/0 conn(0x7fd37415e180 0x7fd3740ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 -- 172.21.15.129:0/2871777880 >> v1:172.21.15.129:6789/0 conn(0x7fd374070c10 legacy=0x7fd37406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 -- 172.21.15.129:0/2871777880 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3740cec60 con 0x7fd374162660 2024-04-25T00:42:56.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd3737fe640 1 -- 172.21.15.129:0/2871777880 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3326775208 0 0) 0x7fd3740ce6a0 con 0x7fd374070c10 2024-04-25T00:42:56.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd378a20640 1 --2- 172.21.15.129:0/2871777880 >> v2:172.21.15.196:3300/0 conn(0x7fd37415e180 0x7fd3740ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:56.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd373fff640 1 --2- 172.21.15.129:0/2871777880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374162660 0x7fd3740706d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd374071290 tx=0x7fd36001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a957cdfee432a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd3737fe640 1 -- 172.21.15.129:0/2871777880 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3600b0070 con 0x7fd374162660 2024-04-25T00:42:56.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd3737fe640 1 -- 172.21.15.129:0/2871777880 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3600ab600 con 0x7fd374162660 2024-04-25T00:42:56.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd3737fe640 1 -- 172.21.15.129:0/2871777880 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3600ab9d0 con 0x7fd374162660 2024-04-25T00:42:56.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- 172.21.15.129:0/2871777880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374162660 msgr2=0x7fd3740706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2871777880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374162660 0x7fd3740706d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd374071290 tx=0x7fd36001c520 comp rx=0 tx=0).stop 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- 172.21.15.129:0/2871777880 shutdown_connections 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2871777880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374162660 0x7fd3740706d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2871777880 >> v2:172.21.15.196:3300/0 conn(0x7fd37415e180 0x7fd3740ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- 172.21.15.129:0/2871777880 >> 172.21.15.129:0/2871777880 conn(0x7fd3740d12c0 msgr2=0x7fd3740d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- 172.21.15.129:0/2871777880 shutdown_connections 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.772+0000 7fd37ac84640 1 -- 172.21.15.129:0/2871777880 wait complete. 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 Processor -- start 2024-04-25T00:42:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- start start 2024-04-25T00:42:56.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 0x7fd37414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd37415e180 0x7fd37414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd374162660 0x7fd37414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd37416b170 con 0x7fd374070c10 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd37416aff0 con 0x7fd37415e180 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd37416b2f0 con 0x7fd374162660 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 0x7fd37414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 0x7fd37414baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52750/0 (socket says 172.21.15.129:52750) 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 -- 172.21.15.129:0/2197205422 learned_addr learned my addr 172.21.15.129:0/2197205422 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd379221640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd374162660 0x7fd37414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 -- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd374162660 msgr2=0x7fd37414f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd374162660 0x7fd37414f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 -- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd37415e180 msgr2=0x7fd37414af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd37415e180 0x7fd37414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 -- 172.21.15.129:0/2197205422 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37414b4b0 con 0x7fd374070c10 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd378a20640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 0x7fd37414baa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c7c0 tx=0x7fd36800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368007a30 con 0x7fd374070c10 2024-04-25T00:42:56.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd368007bd0 con 0x7fd374070c10 2024-04-25T00:42:56.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3680113f0 con 0x7fd374070c10 2024-04-25T00:42:56.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37415d9e0 con 0x7fd374070c10 2024-04-25T00:42:56.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37415d700 con 0x7fd374070c10 2024-04-25T00:42:56.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.776+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd374062e50 con 0x7fd374070c10 2024-04-25T00:42:56.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.780+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd368011590 con 0x7fd374070c10 2024-04-25T00:42:56.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.780+0000 7fd3717fa640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd3580678f0 0x7fd358069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:56.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.780+0000 7fd373fff640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd3580678f0 0x7fd358069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:56.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.780+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fd368085be0 con 0x7fd374070c10 2024-04-25T00:42:56.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.780+0000 7fd373fff640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd3580678f0 0x7fd358069db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ad000 tx=0x7fd360008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:56.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.784+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd368015020 con 0x7fd374070c10 2024-04-25T00:42:56.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:56.944+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd37406f1c0 con 0x7fd374070c10 2024-04-25T00:42:57.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:57.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:42:57.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.624+0000 7fd3717fa640 1 -- 172.21.15.129:0/2197205422 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fd368056990 con 0x7fd374070c10 2024-04-25T00:42:57.628 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' created 2024-04-25T00:42:57.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd3580678f0 msgr2=0x7fd358069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:57.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd3580678f0 0x7fd358069db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ad000 tx=0x7fd360008040 comp rx=0 tx=0).stop 2024-04-25T00:42:57.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 msgr2=0x7fd37414baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 0x7fd37414baa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c7c0 tx=0x7fd36800cc80 comp rx=0 tx=0).stop 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 shutdown_connections 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd3580678f0 0x7fd358069db0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd374162660 0x7fd37414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd37415e180 0x7fd37414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 --2- 172.21.15.129:0/2197205422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd374070c10 0x7fd37414baa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 >> 172.21.15.129:0/2197205422 conn(0x7fd3740d12c0 msgr2=0x7fd3740d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 shutdown_connections 2024-04-25T00:42:57.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.628+0000 7fd37ac84640 1 -- 172.21.15.129:0/2197205422 wait complete. 2024-04-25T00:42:57.668 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:42:57.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.784+0000 7fcdd9350640 1 Processor -- start 2024-04-25T00:42:57.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.784+0000 7fcdd9350640 1 -- start start 2024-04-25T00:42:57.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd9350640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcdd415bd90 0x7fcdd40cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd9350640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd415c170 0x7fcdd40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd9350640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd40ccb80 con 0x7fcdd406f370 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd9350640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40cc930 con 0x7fcdd415bd90 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd9350640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd40cd4b0 con 0x7fcdd415c170 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcdd415bd90 0x7fcdd40cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcdd415bd90 0x7fcdd40cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52762/0 (socket says 172.21.15.129:52762) 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 -- 172.21.15.129:0/735227892 learned_addr learned my addr 172.21.15.129:0/735227892 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdcbfff640 1 --2- 172.21.15.129:0/735227892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd415c170 0x7fcdd40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 -- 172.21.15.129:0/735227892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd415c170 msgr2=0x7fcdd40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 --2- 172.21.15.129:0/735227892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd415c170 0x7fcdd40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 -- 172.21.15.129:0/735227892 >> v1:172.21.15.196:6789/0 conn(0x7fcdd406f370 legacy=0x7fcdd406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:57.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 -- 172.21.15.129:0/735227892 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd40cce60 con 0x7fcdd415bd90 2024-04-25T00:42:57.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd2ffd640 1 --2- 172.21.15.129:0/735227892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415bd90 0x7fcdd40cf060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009870 tx=0x7fcdc401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549034553c783e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:57.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd27fc640 1 -- 172.21.15.129:0/735227892 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40b2070 con 0x7fcdd415bd90 2024-04-25T00:42:57.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd27fc640 1 -- 172.21.15.129:0/735227892 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc40ad680 con 0x7fcdd415bd90 2024-04-25T00:42:57.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.788+0000 7fcdd27fc640 1 -- 172.21.15.129:0/735227892 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40ada50 con 0x7fcdd415bd90 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/735227892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415bd90 msgr2=0x7fcdd40cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 --2- 172.21.15.129:0/735227892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415bd90 0x7fcdd40cf060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009870 tx=0x7fcdc401c4d0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/735227892 shutdown_connections 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 --2- 172.21.15.129:0/735227892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd415c170 0x7fcdd40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 --2- 172.21.15.129:0/735227892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415bd90 0x7fcdd40cf060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/735227892 >> 172.21.15.129:0/735227892 conn(0x7fcdd4067730 msgr2=0x7fcdd40b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/735227892 shutdown_connections 2024-04-25T00:42:57.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/735227892 wait complete. 2024-04-25T00:42:57.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 Processor -- start 2024-04-25T00:42:57.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- start start 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd406f370 0x7fcdd414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdd415bd90 0x7fcdd4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 0x7fcdd4151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd406d7e0 con 0x7fcdd415c170 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd406d660 con 0x7fcdd415bd90 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd406d960 con 0x7fcdd406f370 2024-04-25T00:42:57.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 0x7fcdd4151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 0x7fcdd4151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52772/0 (socket says 172.21.15.129:52772) 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 -- 172.21.15.129:0/4053064568 learned_addr learned my addr 172.21.15.129:0/4053064568 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd2ffd640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd406f370 0x7fcdd414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdcbfff640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdd415bd90 0x7fcdd4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 -- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd406f370 msgr2=0x7fcdd414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd406f370 0x7fcdd414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 -- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdd415bd90 msgr2=0x7fcdd4153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdd415bd90 0x7fcdd4153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 -- 172.21.15.129:0/4053064568 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd4154250 con 0x7fcdd415c170 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdcbfff640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdd415bd90 0x7fcdd4153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd37fe640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 0x7fcdd4151a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c730 tx=0x7fcdcc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc003b80 con 0x7fcdd415c170 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdcc003d20 con 0x7fcdd415c170 2024-04-25T00:42:57.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00aa30 con 0x7fcdd415c170 2024-04-25T00:42:57.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd41501c0 con 0x7fcdd415c170 2024-04-25T00:42:57.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.792+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd4150780 con 0x7fcdd415c170 2024-04-25T00:42:57.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.796+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd4062e50 con 0x7fcdd415c170 2024-04-25T00:42:57.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.796+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcdcc00abd0 con 0x7fcdd415c170 2024-04-25T00:42:57.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.796+0000 7fcdcb7fe640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcdb40678f0 0x7fcdb4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:57.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.796+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fcdcc086420 con 0x7fcdd415c170 2024-04-25T00:42:57.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.796+0000 7fcdd2ffd640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcdb40678f0 0x7fcdb4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:57.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.796+0000 7fcdd2ffd640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcdb40678f0 0x7fcdb4069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40af000 tx=0x7fcdc40a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:57.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.800+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcdcc057130 con 0x7fcdd415c170 2024-04-25T00:42:57.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:57.996+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcdd4150b20 con 0x7fcdd415c170 2024-04-25T00:42:58.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.644+0000 7fcdcb7fe640 1 -- 172.21.15.129:0/4053064568 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fcdcc056a30 con 0x7fcdd415c170 2024-04-25T00:42:58.647 INFO:teuthology.orchestra.run.smithi129.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:42:58.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcdb40678f0 msgr2=0x7fcdb4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcdb40678f0 0x7fcdb4069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40af000 tx=0x7fcdc40a7000 comp rx=0 tx=0).stop 2024-04-25T00:42:58.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 msgr2=0x7fcdd4151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 0x7fcdd4151a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c730 tx=0x7fcdcc00cbf0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 shutdown_connections 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcdb40678f0 0x7fcdb4069db0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdd415c170 0x7fcdd4151a60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdd415bd90 0x7fcdd4153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 --2- 172.21.15.129:0/4053064568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdd406f370 0x7fcdd414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 >> 172.21.15.129:0/4053064568 conn(0x7fcdd4067730 msgr2=0x7fcdd415d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:58.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 shutdown_connections 2024-04-25T00:42:58.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.648+0000 7fcdd9350640 1 -- 172.21.15.129:0/4053064568 wait complete. 2024-04-25T00:42:58.691 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:58.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.804+0000 7faa5c15c640 1 Processor -- start 2024-04-25T00:42:58.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.804+0000 7faa5c15c640 1 -- start start 2024-04-25T00:42:58.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa5c15c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7faa541441b0 0x7faa541627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa5c15c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa54162cf0 con 0x7faa540b6bf0 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa5c15c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa5415eac0 con 0x7faa54144590 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa5c15c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7faa5415f6d0 con 0x7faa541441b0 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7faa541441b0 0x7faa541627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7faa541441b0 0x7faa541627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52782/0 (socket says 172.21.15.129:52782) 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 -- 172.21.15.129:0/2064465314 learned_addr learned my addr 172.21.15.129:0/2064465314 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa5a6f9640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7faa540b6bf0 0x7faa54141ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60124/0 (socket says 172.21.15.129:60124) 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 -- 172.21.15.129:0/2064465314 >> v1:172.21.15.196:6789/0 conn(0x7faa54144590 legacy=0x7faa540b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa58ef6640 1 -- 172.21.15.129:0/2064465314 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757045159 0 0) 0x7faa44001120 con 0x7faa540b6bf0 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 -- 172.21.15.129:0/2064465314 >> v1:172.21.15.129:6789/0 conn(0x7faa540b6bf0 legacy=0x7faa54141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 -- 172.21.15.129:0/2064465314 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5415eda0 con 0x7faa541441b0 2024-04-25T00:42:58.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa59ef8640 1 --2- 172.21.15.129:0/2064465314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa541441b0 0x7faa541627b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009e50 tx=0x7faa4c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57bd00d71446776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:58.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa58ef6640 1 -- 172.21.15.129:0/2064465314 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0a58d0 con 0x7faa541441b0 2024-04-25T00:42:58.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa58ef6640 1 -- 172.21.15.129:0/2064465314 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4c0a5a70 con 0x7faa541441b0 2024-04-25T00:42:58.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.808+0000 7faa58ef6640 1 -- 172.21.15.129:0/2064465314 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0a5e40 con 0x7faa541441b0 2024-04-25T00:42:58.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2064465314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa541441b0 msgr2=0x7faa541627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2064465314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa541441b0 0x7faa541627b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009e50 tx=0x7faa4c01ca00 comp rx=0 tx=0).stop 2024-04-25T00:42:58.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2064465314 shutdown_connections 2024-04-25T00:42:58.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2064465314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa541441b0 0x7faa541627b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2064465314 >> 172.21.15.129:0/2064465314 conn(0x7faa540b52d0 msgr2=0x7faa540b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:58.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2064465314 shutdown_connections 2024-04-25T00:42:58.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2064465314 wait complete. 2024-04-25T00:42:58.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 Processor -- start 2024-04-25T00:42:58.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- start start 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa540b6bf0 0x7faa54153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa541441b0 0x7faa5413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 0x7faa5413e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa54143c10 con 0x7faa54144590 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa54143a90 con 0x7faa541441b0 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa541429d0 con 0x7faa540b6bf0 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 0x7faa5413e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 0x7faa5413e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52798/0 (socket says 172.21.15.129:52798) 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 -- 172.21.15.129:0/2506040261 learned_addr learned my addr 172.21.15.129:0/2506040261 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa59ef8640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa540b6bf0 0x7faa54153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa596f7640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa541441b0 0x7faa5413e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 -- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa540b6bf0 msgr2=0x7faa54153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa540b6bf0 0x7faa54153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 -- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa541441b0 msgr2=0x7faa5413e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa541441b0 0x7faa5413e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 -- 172.21.15.129:0/2506040261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa54155aa0 con 0x7faa54144590 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa596f7640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa541441b0 0x7faa5413e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5a6f9640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 0x7faa5413e570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c750 tx=0x7faa4400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44018070 con 0x7faa54144590 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa440070d0 con 0x7faa54144590 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440074c0 con 0x7faa54144590 2024-04-25T00:42:58.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5413d490 con 0x7faa54144590 2024-04-25T00:42:58.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.812+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5413da50 con 0x7faa54144590 2024-04-25T00:42:58.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.816+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa54062e50 con 0x7faa54144590 2024-04-25T00:42:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.816+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa440047f0 con 0x7faa54144590 2024-04-25T00:42:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.816+0000 7faa527fc640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa400678c0 0x7faa40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.816+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7faa44084f50 con 0x7faa54144590 2024-04-25T00:42:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.816+0000 7faa59ef8640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa400678c0 0x7faa40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.816+0000 7faa59ef8640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa400678c0 0x7faa40069d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0ae000 tx=0x7faa4c0a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:58.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.820+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faa54162cf0 con 0x7faa54144590 2024-04-25T00:42:58.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.980+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa54075320 con 0x7faa54144590 2024-04-25T00:42:58.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa527fc640 1 -- 172.21.15.129:0/2506040261 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7faa44055a80 con 0x7faa54144590 2024-04-25T00:42:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:58.989 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:58.001063+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:58.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa400678c0 msgr2=0x7faa40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa400678c0 0x7faa40069d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0ae000 tx=0x7faa4c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 msgr2=0x7faa5413e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 0x7faa5413e570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c750 tx=0x7faa4400cc10 comp rx=0 tx=0).stop 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 shutdown_connections 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa400678c0 0x7faa40069d80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa54144590 0x7faa5413e570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa541441b0 0x7faa5413e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 --2- 172.21.15.129:0/2506040261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa540b6bf0 0x7faa54153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 >> 172.21.15.129:0/2506040261 conn(0x7faa540b52d0 msgr2=0x7faa54160400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 shutdown_connections 2024-04-25T00:42:58.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:58.984+0000 7faa5c15c640 1 -- 172.21.15.129:0/2506040261 wait complete. 2024-04-25T00:42:59.007 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:42:59.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.116+0000 7f0b1c121640 1 Processor -- start 2024-04-25T00:42:59.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.116+0000 7f0b1c121640 1 -- start start 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1c121640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0b140aa0c0 0x7f0b140ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1c121640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 0x7f0b140cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1c121640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1406d1b0 con 0x7f0b1415bf50 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1c121640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1415ca40 con 0x7f0b140aa0c0 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1c121640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1406d490 con 0x7f0b140aa550 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b19ebd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0b1415bf50 0x7f0b1415c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60132/0 (socket says 172.21.15.129:60132) 2024-04-25T00:42:59.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b19ebd640 1 -- 172.21.15.129:0/2434114368 learned_addr learned my addr 172.21.15.129:0/2434114368 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 0x7f0b140cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b196bc640 1 --2- 172.21.15.129:0/2434114368 >> v2:172.21.15.196:3300/0 conn(0x7f0b140aa0c0 0x7f0b140ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472960915 0 0) 0x7f0b1406d1b0 con 0x7f0b1415bf50 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b00003530 con 0x7f0b1415bf50 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2319923856 0 0) 0x7f0b00003530 con 0x7f0b1415bf50 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 msgr2=0x7f0b140cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 0x7f0b140cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 >> v2:172.21.15.196:3300/0 conn(0x7f0b140aa0c0 msgr2=0x7f0b140ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 --2- 172.21.15.129:0/2434114368 >> v2:172.21.15.196:3300/0 conn(0x7f0b140aa0c0 0x7f0b140ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1406d770 con 0x7f0b1415bf50 2024-04-25T00:42:59.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f0b10001680 con 0x7f0b1415bf50 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 >> v1:172.21.15.129:6789/0 conn(0x7f0b1415bf50 legacy=0x7f0b1415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b000039c0 0x7f0b00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 0x7f0b00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b00004d50 0x7f0b000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415ca40 con 0x7f0b00004d50 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1406d490 con 0x7f0b000039c0 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0000b9d0 con 0x7f0b140aa550 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b00004d50 0x7f0b000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b19ebd640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b000039c0 0x7f0b00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b196bc640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 0x7f0b00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 msgr2=0x7f0b00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b140aa550 0x7f0b00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b000039c0 msgr2=0x7f0b00003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b000039c0 0x7f0b00003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.120+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2434114368 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b00003530 con 0x7f0b00004d50 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b00004d50 0x7f0b000051d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009fa0 tx=0x7f0b0c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da4fb90fe6e5829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0b2070 con 0x7f0b00004d50 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c0a5d10 con 0x7f0b00004d50 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b18ebb640 1 -- 172.21.15.129:0/2434114368 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c009bf0 con 0x7f0b00004d50 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2434114368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b00004d50 msgr2=0x7f0b000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2434114368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b00004d50 0x7f0b000051d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009fa0 tx=0x7f0b0c01cc40 comp rx=0 tx=0).stop 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2434114368 reap_dead start 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2434114368 shutdown_connections 2024-04-25T00:42:59.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2434114368 >> 172.21.15.129:0/2434114368 conn(0x7f0b140b14a0 msgr2=0x7f0b140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:59.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2434114368 shutdown_connections 2024-04-25T00:42:59.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2434114368 wait complete. 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 Processor -- start 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- start start 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b141e5870 0x7f0b14151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b1414fed0 0x7f0b141e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 0x7f0b141e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1406cd60 con 0x7f0b141e5c50 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1406dd30 con 0x7f0b141e5870 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1406cee0 con 0x7f0b1414fed0 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 0x7f0b141e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 0x7f0b141e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52810/0 (socket says 172.21.15.129:52810) 2024-04-25T00:42:59.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2678678979 learned_addr learned my addr 172.21.15.129:0/2678678979 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b196bc640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b1414fed0 0x7f0b141e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b19ebd640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b141e5870 0x7f0b14151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b1414fed0 msgr2=0x7f0b141e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b1414fed0 0x7f0b141e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b141e5870 msgr2=0x7f0b14151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b141e5870 0x7f0b14151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 -- 172.21.15.129:0/2678678979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b14150390 con 0x7f0b141e5c50 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b196bc640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b1414fed0 0x7f0b141e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b19ebd640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b141e5870 0x7f0b14151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1a6be640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 0x7f0b141e42c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c002240 tx=0x7f0b0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0b2040 con 0x7f0b141e5c50 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c0a5b10 con 0x7f0b141e5c50 2024-04-25T00:42:59.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c009bf0 con 0x7f0b141e5c50 2024-04-25T00:42:59.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b14150670 con 0x7f0b141e5c50 2024-04-25T00:42:59.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.124+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b14150c30 con 0x7f0b141e5c50 2024-04-25T00:42:59.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.128+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b14062e50 con 0x7f0b141e5c50 2024-04-25T00:42:59.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.128+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0b0c0c0030 con 0x7f0b141e5c50 2024-04-25T00:42:59.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.128+0000 7f0b0a7fc640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0af80678f0 0x7f0af8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.128+0000 7f0b19ebd640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0af80678f0 0x7f0af8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.128+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f0b0c120490 con 0x7f0b141e5c50 2024-04-25T00:42:59.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.128+0000 7f0b19ebd640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0af80678f0 0x7f0af8069db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b10001ec0 tx=0x7f0b10005d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:59.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.132+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0b0c0c7060 con 0x7f0b141e5c50 2024-04-25T00:42:59.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.292+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b14075320 con 0x7f0b141e5c50 2024-04-25T00:42:59.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b0a7fc640 1 -- 172.21.15.129:0/2678678979 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16654 (secure 0 0 0) 0x7f0b0c016950 con 0x7f0b141e5c50 2024-04-25T00:42:59.300 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:42:59.300 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":19,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:58.001063+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0af80678f0 msgr2=0x7f0af8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0af80678f0 0x7f0af8069db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b10001ec0 tx=0x7f0b10005d50 comp rx=0 tx=0).stop 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 msgr2=0x7f0b141e42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 0x7f0b141e42c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c002240 tx=0x7f0b0c009750 comp rx=0 tx=0).stop 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 shutdown_connections 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0af80678f0 0x7f0af8069db0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b1414fed0 0x7f0b141e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b141e5c50 0x7f0b141e42c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 --2- 172.21.15.129:0/2678678979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b141e5870 0x7f0b14151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 >> 172.21.15.129:0/2678678979 conn(0x7f0b140b14a0 msgr2=0x7f0b1415c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:59.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 shutdown_connections 2024-04-25T00:42:59.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.296+0000 7f0b1c121640 1 -- 172.21.15.129:0/2678678979 wait complete. 2024-04-25T00:42:59.319 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:42:59.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.436+0000 7fcfd63d4640 1 Processor -- start 2024-04-25T00:42:59.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.436+0000 7fcfd63d4640 1 -- start start 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd63d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd63d4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcfd00cfef0 0x7fcfd015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd63d4640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd00b3370 con 0x7fcfd00b2e70 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd63d4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd00b3650 con 0x7fcfd00cfef0 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd63d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd00b3930 con 0x7fcfd015c9b0 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fcfd00b2e70 0x7fcfd015de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60148/0 (socket says 172.21.15.129:60148) 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 -- 172.21.15.129:0/459530349 learned_addr learned my addr 172.21.15.129:0/459530349 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfcffff640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfcf7fe640 1 --2- 172.21.15.129:0/459530349 >> v2:172.21.15.196:3300/0 conn(0x7fcfd00cfef0 0x7fcfd015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2490589338 0 0) 0x7fcfd00b3370 con 0x7fcfd00b2e70 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfb8003530 con 0x7fcfd00b2e70 2024-04-25T00:42:59.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3547357431 0 0) 0x7fcfb8003530 con 0x7fcfd00b2e70 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 msgr2=0x7fcfd00cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 0x7fcfd00cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 >> v2:172.21.15.196:3300/0 conn(0x7fcfd00cfef0 msgr2=0x7fcfd015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 --2- 172.21.15.129:0/459530349 >> v2:172.21.15.196:3300/0 conn(0x7fcfd00cfef0 0x7fcfd015cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00b3c10 con 0x7fcfd00b2e70 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fcfc8001a00 con 0x7fcfd00b2e70 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 >> v1:172.21.15.129:6789/0 conn(0x7fcfd00b2e70 legacy=0x7fcfd015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 0x7fcfb80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfb8003f40 0x7fcfb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00b3650 con 0x7fcfb8004d50 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00b3930 con 0x7fcfb8003f40 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb800b9d0 con 0x7fcfd015c9b0 2024-04-25T00:42:59.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfcf7fe640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfb8003f40 0x7fcfb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 -- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 msgr2=0x7fcfb80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd015c9b0 0x7fcfb80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 -- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfb8003f40 msgr2=0x7fcfb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfb8003f40 0x7fcfb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.440+0000 7fcfd4971640 1 -- 172.21.15.129:0/459530349 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb8003530 con 0x7fcfb8004d50 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0068a30 tx=0x7fcfc801d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2df3491ecac388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800a4c0 con 0x7fcfb8004d50 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc80ad630 con 0x7fcfb8004d50 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfceffd640 1 -- 172.21.15.129:0/459530349 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80ada20 con 0x7fcfb8004d50 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- 172.21.15.129:0/459530349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb8004d50 msgr2=0x7fcfb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/459530349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb8004d50 0x7fcfb80051d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0068a30 tx=0x7fcfc801d2e0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 -- 172.21.15.129:0/459530349 reap_dead start 2024-04-25T00:42:59.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- 172.21.15.129:0/459530349 shutdown_connections 2024-04-25T00:42:59.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- 172.21.15.129:0/459530349 >> 172.21.15.129:0/459530349 conn(0x7fcfd0071ac0 msgr2=0x7fcfd0071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:59.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- 172.21.15.129:0/459530349 shutdown_connections 2024-04-25T00:42:59.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- 172.21.15.129:0/459530349 wait complete. 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 Processor -- start 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- start start 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd014f0e0 0x7fcfd00adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd00ae530 0x7fcfd00aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 0x7fcfd006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd015d450 con 0x7fcfd00abfb0 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd015d2d0 con 0x7fcfd00ae530 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd00ad880 con 0x7fcfd014f0e0 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 0x7fcfd006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 0x7fcfd006d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52834/0 (socket says 172.21.15.129:52834) 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 -- 172.21.15.129:0/3514388035 learned_addr learned my addr 172.21.15.129:0/3514388035 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 -- 172.21.15.129:0/3514388035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd014f0e0 msgr2=0x7fcfd00adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd014f0e0 0x7fcfd00adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 -- 172.21.15.129:0/3514388035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd00ae530 msgr2=0x7fcfd00aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd00ae530 0x7fcfd00aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 -- 172.21.15.129:0/3514388035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00ac7e0 con 0x7fcfd00abfb0 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd4971640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 0x7fcfd006d350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcfc801d7c0 tx=0x7fcfb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80b2040 con 0x7fcfd00abfb0 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc80017c0 con 0x7fcfd00abfb0 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.444+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd00acac0 con 0x7fcfd00abfb0 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800a4c0 con 0x7fcfd00abfb0 2024-04-25T00:42:59.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd00ad080 con 0x7fcfd00abfb0 2024-04-25T00:42:59.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd0062e50 con 0x7fcfd00abfb0 2024-04-25T00:42:59.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcfc80ad630 con 0x7fcfd00abfb0 2024-04-25T00:42:59.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfccff9640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcfbc0678c0 0x7fcfbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:59.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fcfc811fc90 con 0x7fcfd00abfb0 2024-04-25T00:42:59.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfcffff640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcfbc0678c0 0x7fcfbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:59.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.448+0000 7fcfcffff640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcfbc0678c0 0x7fcfbc069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40099f0 tx=0x7fcfc4004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:59.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.452+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcfc8017050 con 0x7fcfd00abfb0 2024-04-25T00:42:59.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:42:59.652+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcfd0075320 con 0x7fcfd00abfb0 2024-04-25T00:43:00.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfccff9640 1 -- 172.21.15.129:0/3514388035 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fcfc80f07d0 con 0x7fcfd00abfb0 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcfbc0678c0 msgr2=0x7fcfbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcfbc0678c0 0x7fcfbc069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40099f0 tx=0x7fcfc4004740 comp rx=0 tx=0).stop 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 msgr2=0x7fcfd006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 0x7fcfd006d350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcfc801d7c0 tx=0x7fcfb8005840 comp rx=0 tx=0).stop 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 shutdown_connections 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcfbc0678c0 0x7fcfbc069d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfd00abfb0 0x7fcfd006d350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd00ae530 0x7fcfd00aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 --2- 172.21.15.129:0/3514388035 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd014f0e0 0x7fcfd00adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 >> 172.21.15.129:0/3514388035 conn(0x7fcfd0071ac0 msgr2=0x7fcfd0158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:00.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 shutdown_connections 2024-04-25T00:43:00.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.648+0000 7fcfd63d4640 1 -- 172.21.15.129:0/3514388035 wait complete. 2024-04-25T00:43:00.673 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:43:00.674 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:43:00.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.788+0000 7fa4173ce640 1 Processor -- start 2024-04-25T00:43:00.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.788+0000 7fa4173ce640 1 -- start start 2024-04-25T00:43:00.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa4173ce640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 0x7fa410166f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa4173ce640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41006dbf0 con 0x7fa41015c590 2024-04-25T00:43:00.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa4173ce640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41006f090 con 0x7fa410145960 2024-04-25T00:43:00.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa4173ce640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa41006cfe0 con 0x7fa4100ccea0 2024-04-25T00:43:00.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa414969640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fa41015c590 0x7fa4100cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60164/0 (socket says 172.21.15.129:60164) 2024-04-25T00:43:00.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa414969640 1 -- 172.21.15.129:0/349863714 learned_addr learned my addr 172.21.15.129:0/349863714 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:00.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 0x7fa410166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2227478940 0 0) 0x7fa41006dbf0 con 0x7fa41015c590 2024-04-25T00:43:00.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3fc003530 con 0x7fa41015c590 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3585527039 0 0) 0x7fa3fc003530 con 0x7fa41015c590 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 msgr2=0x7fa410166f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 0x7fa410166f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 >> v1:172.21.15.196:6789/0 conn(0x7fa410145960 legacy=0x7fa410160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41006d2c0 con 0x7fa41015c590 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fa400001680 con 0x7fa41015c590 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 >> v1:172.21.15.129:6789/0 conn(0x7fa41015c590 legacy=0x7fa4100cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc0039f0 0x7fa3fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 0x7fa3fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3fc004d50 0x7fa3fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa41006cfe0 con 0x7fa3fc004d50 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc00bb30 con 0x7fa3fc0039f0 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc00bd30 con 0x7fa4100ccea0 2024-04-25T00:43:00.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3fc004d50 0x7fa3fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa414969640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 0x7fa3fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41516a640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc0039f0 0x7fa3fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 -- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 msgr2=0x7fa3fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4100ccea0 0x7fa3fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 -- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc0039f0 msgr2=0x7fa3fc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc0039f0 0x7fa3fc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.792+0000 7fa41596b640 1 -- 172.21.15.129:0/349863714 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc003530 con 0x7fa3fc004d50 2024-04-25T00:43:00.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa41596b640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3fc004d50 0x7fa3fc0051d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4100cd6a0 tx=0x7fa40801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfd85f312d720e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:00.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080b2070 con 0x7fa3fc004d50 2024-04-25T00:43:00.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4080a5b20 con 0x7fa3fc004d50 2024-04-25T00:43:00.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa407fff640 1 -- 172.21.15.129:0/349863714 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408009c10 con 0x7fa3fc004d50 2024-04-25T00:43:00.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- 172.21.15.129:0/349863714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3fc004d50 msgr2=0x7fa3fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 --2- 172.21.15.129:0/349863714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3fc004d50 0x7fa3fc0051d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4100cd6a0 tx=0x7fa40801cc20 comp rx=0 tx=0).stop 2024-04-25T00:43:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa41596b640 1 -- 172.21.15.129:0/349863714 reap_dead start 2024-04-25T00:43:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- 172.21.15.129:0/349863714 shutdown_connections 2024-04-25T00:43:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- 172.21.15.129:0/349863714 >> 172.21.15.129:0/349863714 conn(0x7fa410067730 msgr2=0x7fa4100b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- 172.21.15.129:0/349863714 shutdown_connections 2024-04-25T00:43:00.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- 172.21.15.129:0/349863714 wait complete. 2024-04-25T00:43:00.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 Processor -- start 2024-04-25T00:43:00.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- start start 2024-04-25T00:43:00.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc004d50 0x7fa4100ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 0x7fa41015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa41013cdf0 0x7fa41015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100cf890 con 0x7fa41013c970 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa41006d880 con 0x7fa3fc004d50 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa4173ce640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa41006cc40 con 0x7fa41013cdf0 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 0x7fa41015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 0x7fa41015b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52850/0 (socket says 172.21.15.129:52850) 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 -- 172.21.15.129:0/1686349626 learned_addr learned my addr 172.21.15.129:0/1686349626 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa41516a640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc004d50 0x7fa4100ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa41596b640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa41013cdf0 0x7fa41015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 -- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa41013cdf0 msgr2=0x7fa41015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa41013cdf0 0x7fa41015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 -- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc004d50 msgr2=0x7fa4100ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc004d50 0x7fa4100ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:00.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.796+0000 7fa414969640 1 -- 172.21.15.129:0/1686349626 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41013bbb0 con 0x7fa41013c970 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa414969640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 0x7fa41015b370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa400011820 tx=0x7fa400011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400010040 con 0x7fa41013c970 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40001a070 con 0x7fa41013c970 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40000fde0 con 0x7fa41013c970 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41013be90 con 0x7fa41013c970 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4100ae580 con 0x7fa41013c970 2024-04-25T00:43:00.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410062e50 con 0x7fa41013c970 2024-04-25T00:43:00.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa40001f460 con 0x7fa41013c970 2024-04-25T00:43:00.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa405ffb640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa3f40678c0 0x7fa3f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:00.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa400088ec0 con 0x7fa41013c970 2024-04-25T00:43:00.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa41516a640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa3f40678c0 0x7fa3f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:00.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.800+0000 7fa41516a640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa3f40678c0 0x7fa3f4069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa410144ae0 tx=0x7fa40c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:00.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:00.804+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa400059a10 con 0x7fa41013c970 2024-04-25T00:43:01.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.000+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa410075320 con 0x7fa41013c970 2024-04-25T00:43:01.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa405ffb640 1 -- 172.21.15.129:0/1686349626 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fa400059310 con 0x7fa41013c970 2024-04-25T00:43:01.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa3f40678c0 msgr2=0x7fa3f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:01.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa3f40678c0 0x7fa3f4069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa410144ae0 tx=0x7fa40c009040 comp rx=0 tx=0).stop 2024-04-25T00:43:01.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 msgr2=0x7fa41015b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:01.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 0x7fa41015b370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa400011820 tx=0x7fa400011ce0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 shutdown_connections 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa3f40678c0 0x7fa3f4069d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa41013cdf0 0x7fa41015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41013c970 0x7fa41015b370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 --2- 172.21.15.129:0/1686349626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3fc004d50 0x7fa4100ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 >> 172.21.15.129:0/1686349626 conn(0x7fa410067730 msgr2=0x7fa4100b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.676+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 shutdown_connections 2024-04-25T00:43:01.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.680+0000 7fa4173ce640 1 -- 172.21.15.129:0/1686349626 wait complete. 2024-04-25T00:43:01.701 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:43:01.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:43:01.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 Processor -- start 2024-04-25T00:43:01.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 -- start start 2024-04-25T00:43:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f323415d190 0x7f3234072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3234070fb0 0x7f32340ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32340715d0 con 0x7f3234072cb0 2024-04-25T00:43:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f32340718b0 con 0x7f3234070fb0 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f323aaba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3234071b90 con 0x7f323415d190 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3234070fb0 0x7f32340ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3234070fb0 0x7f32340ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52858/0 (socket says 172.21.15.129:52858) 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 -- 172.21.15.129:0/73232521 learned_addr learned my addr 172.21.15.129:0/73232521 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3238856640 1 --2- 172.21.15.129:0/73232521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f323415d190 0x7f3234072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 -- 172.21.15.129:0/73232521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f323415d190 msgr2=0x7f3234072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 --2- 172.21.15.129:0/73232521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f323415d190 0x7f3234072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 -- 172.21.15.129:0/73232521 >> v1:172.21.15.196:6789/0 conn(0x7f3234072cb0 legacy=0x7f32340708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.824+0000 7f3239057640 1 -- 172.21.15.129:0/73232521 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3234071e70 con 0x7f3234070fb0 2024-04-25T00:43:01.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 --2- 172.21.15.129:0/73232521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3234070fb0 0x7f32340ced50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f322c009e50 tx=0x7f322c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0ccb39e7b22da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:01.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f322b7fe640 1 -- 172.21.15.129:0/73232521 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c0a5670 con 0x7f3234070fb0 2024-04-25T00:43:01.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f322b7fe640 1 -- 172.21.15.129:0/73232521 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c0a5810 con 0x7f3234070fb0 2024-04-25T00:43:01.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f322b7fe640 1 -- 172.21.15.129:0/73232521 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c0a5be0 con 0x7f3234070fb0 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- 172.21.15.129:0/73232521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3234070fb0 msgr2=0x7f32340ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 --2- 172.21.15.129:0/73232521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3234070fb0 0x7f32340ced50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f322c009e50 tx=0x7f322c01c9f0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- 172.21.15.129:0/73232521 shutdown_connections 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 --2- 172.21.15.129:0/73232521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3234070fb0 0x7f32340ced50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 --2- 172.21.15.129:0/73232521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f323415d190 0x7f3234072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- 172.21.15.129:0/73232521 >> 172.21.15.129:0/73232521 conn(0x7f32340b2840 msgr2=0x7f32340b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- 172.21.15.129:0/73232521 shutdown_connections 2024-04-25T00:43:01.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- 172.21.15.129:0/73232521 wait complete. 2024-04-25T00:43:01.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 Processor -- start 2024-04-25T00:43:01.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- start start 2024-04-25T00:43:01.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3234070fb0 0x7f3234156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:01.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3234072cb0 0x7f323414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:01.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 0x7f32340b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:01.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32340cfeb0 con 0x7f323415d190 2024-04-25T00:43:01.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32340cfd30 con 0x7f3234072cb0 2024-04-25T00:43:01.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f323aaba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f32340d0030 con 0x7f3234070fb0 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 0x7f32340b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 0x7f32340b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52862/0 (socket says 172.21.15.129:52862) 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 -- 172.21.15.129:0/1819633609 learned_addr learned my addr 172.21.15.129:0/1819633609 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3238856640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3234070fb0 0x7f3234156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f322bfff640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3234072cb0 0x7f323414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 -- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3234070fb0 msgr2=0x7f3234156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3234070fb0 0x7f3234156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 -- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3234072cb0 msgr2=0x7f323414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3234072cb0 0x7f323414ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 -- 172.21.15.129:0/1819633609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323414dcd0 con 0x7f323415d190 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.828+0000 7f3239057640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 0x7f32340b0480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f322c01b0a0 tx=0x7f322c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c009510 con 0x7f323415d190 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c0be070 con 0x7f323415d190 2024-04-25T00:43:01.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323414dfb0 con 0x7f323415d190 2024-04-25T00:43:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c0b8890 con 0x7f323415d190 2024-04-25T00:43:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323414e570 con 0x7f323415d190 2024-04-25T00:43:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3234062e50 con 0x7f323415d190 2024-04-25T00:43:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f322c0b8a30 con 0x7f323415d190 2024-04-25T00:43:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f32297fa640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f32200678c0 0x7f3220069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:01.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f322c120ec0 con 0x7f323415d190 2024-04-25T00:43:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f3238856640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f32200678c0 0x7f3220069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:01.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.832+0000 7f3238856640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f32200678c0 0x7f3220069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f323000b930 tx=0x7f3230008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:01.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:01.836+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f322c0165e0 con 0x7f323415d190 2024-04-25T00:43:02.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.032+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3234146ab0 con 0x7f323415d190 2024-04-25T00:43:02.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f32297fa640 1 -- 172.21.15.129:0/1819633609 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f322c0f1a80 con 0x7f323415d190 2024-04-25T00:43:02.039 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:02.039 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:42:58.001381+0000","modified":"2024-04-25T00:43:01.006515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-25T00:43:02.039 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f32200678c0 msgr2=0x7f3220069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f32200678c0 0x7f3220069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f323000b930 tx=0x7f3230008040 comp rx=0 tx=0).stop 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 msgr2=0x7f32340b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 0x7f32340b0480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f322c01b0a0 tx=0x7f322c019790 comp rx=0 tx=0).stop 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 shutdown_connections 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f32200678c0 0x7f3220069d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323415d190 0x7f32340b0480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3234072cb0 0x7f323414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 --2- 172.21.15.129:0/1819633609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3234070fb0 0x7f3234156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 >> 172.21.15.129:0/1819633609 conn(0x7f32340b2840 msgr2=0x7f32340708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:02.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 shutdown_connections 2024-04-25T00:43:02.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.036+0000 7f323aaba640 1 -- 172.21.15.129:0/1819633609 wait complete. 2024-04-25T00:43:02.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:43:02.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.172+0000 7f98f103a640 1 Processor -- start 2024-04-25T00:43:02.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.172+0000 7f98f103a640 1 -- start start 2024-04-25T00:43:02.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f98ec15bf50 0x7f98ec15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec06d1b0 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec06d490 con 0x7f98ec0cd260 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f98ec06d770 con 0x7f98ec15bf50 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ead9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f98ec15bf50 0x7f98ec15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ea59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f98ec0aa0c0 0x7f98ec0ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60180/0 (socket says 172.21.15.129:60180) 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ea59c640 1 -- 172.21.15.129:0/3214908931 learned_addr learned my addr 172.21.15.129:0/3214908931 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ead9d640 1 -- 172.21.15.129:0/3214908931 >> v1:172.21.15.196:6789/0 conn(0x7f98ec0cd260 legacy=0x7f98ec06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ead9d640 1 -- 172.21.15.129:0/3214908931 >> v1:172.21.15.129:6789/0 conn(0x7f98ec0aa0c0 legacy=0x7f98ec0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ead9d640 1 -- 172.21.15.129:0/3214908931 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec06da50 con 0x7f98ec15bf50 2024-04-25T00:43:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98ead9d640 1 --2- 172.21.15.129:0/3214908931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec15bf50 0x7f98ec15c330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009d90 tx=0x7f98e401c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1ad0d2169d2a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98e9d9b640 1 -- 172.21.15.129:0/3214908931 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40b2070 con 0x7f98ec15bf50 2024-04-25T00:43:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98e9d9b640 1 -- 172.21.15.129:0/3214908931 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e40a5860 con 0x7f98ec15bf50 2024-04-25T00:43:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98e9d9b640 1 -- 172.21.15.129:0/3214908931 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40a5c30 con 0x7f98ec15bf50 2024-04-25T00:43:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 -- 172.21.15.129:0/3214908931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec15bf50 msgr2=0x7f98ec15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 --2- 172.21.15.129:0/3214908931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec15bf50 0x7f98ec15c330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009d90 tx=0x7f98e401c7c0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 -- 172.21.15.129:0/3214908931 shutdown_connections 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 --2- 172.21.15.129:0/3214908931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec15bf50 0x7f98ec15c330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.176+0000 7f98f103a640 1 -- 172.21.15.129:0/3214908931 >> 172.21.15.129:0/3214908931 conn(0x7f98ec0b14a0 msgr2=0x7f98ec0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- 172.21.15.129:0/3214908931 shutdown_connections 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- 172.21.15.129:0/3214908931 wait complete. 2024-04-25T00:43:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 Processor -- start 2024-04-25T00:43:02.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- start start 2024-04-25T00:43:02.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 0x7f98ec1f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec0cd260 0x7f98ec156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98ec14b6b0 0x7f98ec1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec15c9f0 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec15c870 con 0x7f98ec0cd260 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec06cd60 con 0x7f98ec14b6b0 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 0x7f98ec1f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 0x7f98ec1f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52880/0 (socket says 172.21.15.129:52880) 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 -- 172.21.15.129:0/1173083754 learned_addr learned my addr 172.21.15.129:0/1173083754 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98eb59e640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98ec14b6b0 0x7f98ec1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ea59c640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec0cd260 0x7f98ec156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 -- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98ec14b6b0 msgr2=0x7f98ec1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98ec14b6b0 0x7f98ec1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 -- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec0cd260 msgr2=0x7f98ec156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec0cd260 0x7f98ec156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 -- 172.21.15.129:0/1173083754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec1f7f90 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98ead9d640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 0x7f98ec1f41e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98e40193d0 tx=0x7f98e4019560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40b2040 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e40bf070 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec1f8230 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40add90 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec1f8d10 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.180+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec062e50 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.184+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f98e40b8470 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.184+0000 7f98e37fe640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f98d40678f0 0x7f98d4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.184+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f98e4120b90 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.184+0000 7f98ea59c640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f98d40678f0 0x7f98d4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.184+0000 7f98ea59c640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f98d40678f0 0x7f98d4069db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009fd0 tx=0x7f98d800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.188+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f98e40c8020 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.348+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98ec075320 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98e37fe640 1 -- 172.21.15.129:0/1173083754 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f98e4016480 con 0x7f98ec0aa0c0 2024-04-25T00:43:02.401 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:02.401 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:59.648406+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:43:02.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f98d40678f0 msgr2=0x7f98d4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f98d40678f0 0x7f98d4069db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009fd0 tx=0x7f98d800b040 comp rx=0 tx=0).stop 2024-04-25T00:43:02.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 msgr2=0x7f98ec1f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 0x7f98ec1f41e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98e40193d0 tx=0x7f98e4019560 comp rx=0 tx=0).stop 2024-04-25T00:43:02.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 shutdown_connections 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f98d40678f0 0x7f98d4069db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98ec14b6b0 0x7f98ec1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec0cd260 0x7f98ec156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 --2- 172.21.15.129:0/1173083754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98ec0aa0c0 0x7f98ec1f41e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 >> 172.21.15.129:0/1173083754 conn(0x7f98ec0b14a0 msgr2=0x7f98ec15aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 shutdown_connections 2024-04-25T00:43:02.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.352+0000 7f98f103a640 1 -- 172.21.15.129:0/1173083754 wait complete. 2024-04-25T00:43:02.404 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:43:02.405 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:43:02.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f424d6640 1 Processor -- start 2024-04-25T00:43:02.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f424d6640 1 -- start start 2024-04-25T00:43:02.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f424d6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f3c0b40d0 0x7f6f3c15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f424d6640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f3c141c70 con 0x7f6f3c070850 2024-04-25T00:43:02.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f424d6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f3c141f50 con 0x7f6f3c13f240 2024-04-25T00:43:02.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f424d6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3c142230 con 0x7f6f3c0b40d0 2024-04-25T00:43:02.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3b7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f6f3c070850 0x7f6f3c0b39c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60186/0 (socket says 172.21.15.129:60186) 2024-04-25T00:43:02.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3b7fe640 1 -- 172.21.15.129:0/4228877309 learned_addr learned my addr 172.21.15.129:0/4228877309 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:02.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f40a73640 1 --2- 172.21.15.129:0/4228877309 >> v2:172.21.15.196:3300/0 conn(0x7f6f3c0b40d0 0x7f6f3c15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 958239226 0 0) 0x7f6f3c141c70 con 0x7f6f3c070850 2024-04-25T00:43:02.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f24003530 con 0x7f6f3c070850 2024-04-25T00:43:02.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 484004059 0 0) 0x7f6f24003530 con 0x7f6f3c070850 2024-04-25T00:43:02.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 >> v2:172.21.15.196:3300/0 conn(0x7f6f3c0b40d0 msgr2=0x7f6f3c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 --2- 172.21.15.129:0/4228877309 >> v2:172.21.15.196:3300/0 conn(0x7f6f3c0b40d0 0x7f6f3c15ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 >> v1:172.21.15.196:6789/0 conn(0x7f6f3c13f240 legacy=0x7f6f3c13f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c142510 con 0x7f6f3c070850 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f6f30001680 con 0x7f6f3c070850 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 >> v1:172.21.15.129:6789/0 conn(0x7f6f3c070850 legacy=0x7f6f3c0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f240039c0 0x7f6f24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24004810 0x7f6f24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240051d0 0x7f6f2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c142230 con 0x7f6f240039c0 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2400c070 con 0x7f6f24004810 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2400c270 con 0x7f6f240051d0 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f240039c0 0x7f6f24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3b7fe640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24004810 0x7f6f24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3bfff640 1 -- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240051d0 msgr2=0x7f6f2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:02.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240051d0 0x7f6f2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3bfff640 1 -- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24004810 msgr2=0x7f6f24004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:02.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24004810 0x7f6f24004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f3bfff640 1 -- 172.21.15.129:0/4228877309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f24003530 con 0x7f6f240039c0 2024-04-25T00:43:02.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.520+0000 7f6f40a73640 1 -- 172.21.15.129:0/4228877309 reap_dead start 2024-04-25T00:43:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f240039c0 0x7f6f24003de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c001780 tx=0x7f6f2c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593bd718ea5f5a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0b0070 con 0x7f6f240039c0 2024-04-25T00:43:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0052f0 con 0x7f6f240039c0 2024-04-25T00:43:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f3affd640 1 -- 172.21.15.129:0/4228877309 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0013a0 con 0x7f6f240039c0 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- 172.21.15.129:0/4228877309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f240039c0 msgr2=0x7f6f24003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f240039c0 0x7f6f24003de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c001780 tx=0x7f6f2c01c420 comp rx=0 tx=0).stop 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- 172.21.15.129:0/4228877309 shutdown_connections 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 --2- 172.21.15.129:0/4228877309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f240039c0 0x7f6f24003de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- 172.21.15.129:0/4228877309 >> 172.21.15.129:0/4228877309 conn(0x7f6f3c0ce400 msgr2=0x7f6f3c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- 172.21.15.129:0/4228877309 shutdown_connections 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- 172.21.15.129:0/4228877309 wait complete. 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 Processor -- start 2024-04-25T00:43:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- start start 2024-04-25T00:43:02.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240039c0 0x7f6f3c13e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 0x7f6f3c157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f3c15cf70 0x7f6f3c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c0b3d50 con 0x7f6f3c157660 2024-04-25T00:43:02.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c0b3bd0 con 0x7f6f3c15cf70 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f424d6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3c0b3ed0 con 0x7f6f240039c0 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f3b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 0x7f6f3c157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.524+0000 7f6f3b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 0x7f6f3c157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52910/0 (socket says 172.21.15.129:52910) 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 -- 172.21.15.129:0/2747554443 learned_addr learned my addr 172.21.15.129:0/2747554443 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f40a73640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f3c15cf70 0x7f6f3c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240039c0 0x7f6f3c13e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 -- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240039c0 msgr2=0x7f6f3c13e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240039c0 0x7f6f3c13e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 -- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f3c15cf70 msgr2=0x7f6f3c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f3c15cf70 0x7f6f3c15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 -- 172.21.15.129:0/2747554443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c15dce0 con 0x7f6f3c157660 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3b7fe640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 0x7f6f3c157ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000e780 tx=0x7f6f3000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000ee30 con 0x7f6f3c157660 2024-04-25T00:43:02.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f30018070 con 0x7f6f3c157660 2024-04-25T00:43:02.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c15dfc0 con 0x7f6f3c157660 2024-04-25T00:43:02.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000cb10 con 0x7f6f3c157660 2024-04-25T00:43:02.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c15e580 con 0x7f6f3c157660 2024-04-25T00:43:02.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3c062e50 con 0x7f6f3c157660 2024-04-25T00:43:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6f3000ccb0 con 0x7f6f3c157660 2024-04-25T00:43:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f38ff9640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6f280678c0 0x7f6f28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6f280678c0 0x7f6f28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f30086490 con 0x7f6f3c157660 2024-04-25T00:43:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.528+0000 7f6f3bfff640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6f280678c0 0x7f6f28069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c001750 tx=0x7f6f2c0011a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.532+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6f30057140 con 0x7f6f3c157660 2024-04-25T00:43:02.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.684+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f3c15e8f0 con 0x7f6f280678c0 2024-04-25T00:43:02.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.692+0000 7f6f38ff9640 1 -- 172.21.15.129:0/2747554443 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f6f3c15e8f0 con 0x7f6f280678c0 2024-04-25T00:43:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:02.701 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:43:02.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6f280678c0 msgr2=0x7f6f28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6f280678c0 0x7f6f28069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c001750 tx=0x7f6f2c0011a0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 msgr2=0x7f6f3c157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 0x7f6f3c157ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000e780 tx=0x7f6f3000ec40 comp rx=0 tx=0).stop 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 shutdown_connections 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6f280678c0 0x7f6f28069d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f3c15cf70 0x7f6f3c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f3c157660 0x7f6f3c157ae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 --2- 172.21.15.129:0/2747554443 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f240039c0 0x7f6f3c13e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 >> 172.21.15.129:0/2747554443 conn(0x7f6f3c0ce400 msgr2=0x7f6f3c074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 shutdown_connections 2024-04-25T00:43:02.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.696+0000 7f6f424d6640 1 -- 172.21.15.129:0/2747554443 wait complete. 2024-04-25T00:43:02.715 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T00:43:02.605779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.980665"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016159+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:43:00.016159+0000","last_peered":"2024-04-25T00:43:00.016159+0000","last_clean":"2024-04-25T00:43:00.016159+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:43:00.016159+0000","last_undegraded":"2024-04-25T00:43:00.016159+0000","last_fullsized":"2024-04-25T00:43:00.016159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657681+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:42:59.657681+0000","last_peered":"2024-04-25T00:42:59.657681+0000","last_clean":"2024-04-25T00:42:59.657681+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:42:59.657681+0000","last_undegraded":"2024-04-25T00:42:59.657681+0000","last_fullsized":"2024-04-25T00:42:59.657681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014295+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:43:00.014295+0000","last_peered":"2024-04-25T00:43:00.014295+0000","last_clean":"2024-04-25T00:43:00.014295+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:43:00.014295+0000","last_undegraded":"2024-04-25T00:43:00.014295+0000","last_fullsized":"2024-04-25T00:43:00.014295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013425+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:43:00.013425+0000","last_peered":"2024-04-25T00:43:00.013425+0000","last_clean":"2024-04-25T00:43:00.013425+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:43:00.013425+0000","last_undegraded":"2024-04-25T00:43:00.013425+0000","last_fullsized":"2024-04-25T00:43:00.013425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014816+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:43:00.014816+0000","last_peered":"2024-04-25T00:43:00.014816+0000","last_clean":"2024-04-25T00:43:00.014816+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:43:00.014816+0000","last_undegraded":"2024-04-25T00:43:00.014816+0000","last_fullsized":"2024-04-25T00:43:00.014816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013220+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:43:00.013220+0000","last_peered":"2024-04-25T00:43:00.013220+0000","last_clean":"2024-04-25T00:43:00.013220+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:43:00.013220+0000","last_undegraded":"2024-04-25T00:43:00.013220+0000","last_fullsized":"2024-04-25T00:43:00.013220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013356+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:43:00.013356+0000","last_peered":"2024-04-25T00:43:00.013356+0000","last_clean":"2024-04-25T00:43:00.013356+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:43:00.013356+0000","last_undegraded":"2024-04-25T00:43:00.013356+0000","last_fullsized":"2024-04-25T00:43:00.013356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940441+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:42:59.940441+0000","last_peered":"2024-04-25T00:42:59.940441+0000","last_clean":"2024-04-25T00:42:59.940441+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:42:59.940441+0000","last_undegraded":"2024-04-25T00:42:59.940441+0000","last_fullsized":"2024-04-25T00:42:59.940441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016200+0000","last_change":"2024-04-25T00:42:58.698697+0000","last_active":"2024-04-25T00:43:00.016200+0000","last_peered":"2024-04-25T00:43:00.016200+0000","last_clean":"2024-04-25T00:43:00.016200+0000","last_became_active":"2024-04-25T00:42:58.698442+0000","last_became_peered":"2024-04-25T00:42:58.698442+0000","last_unstale":"2024-04-25T00:43:00.016200+0000","last_undegraded":"2024-04-25T00:43:00.016200+0000","last_fullsized":"2024-04-25T00:43:00.016200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663628+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:42:59.663628+0000","last_peered":"2024-04-25T00:42:59.663628+0000","last_clean":"2024-04-25T00:42:59.663628+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:42:59.663628+0000","last_undegraded":"2024-04-25T00:42:59.663628+0000","last_fullsized":"2024-04-25T00:42:59.663628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663300+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:42:59.663300+0000","last_peered":"2024-04-25T00:42:59.663300+0000","last_clean":"2024-04-25T00:42:59.663300+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:42:59.663300+0000","last_undegraded":"2024-04-25T00:42:59.663300+0000","last_fullsized":"2024-04-25T00:42:59.663300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659638+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:42:59.659638+0000","last_peered":"2024-04-25T00:42:59.659638+0000","last_clean":"2024-04-25T00:42:59.659638+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:42:59.659638+0000","last_undegraded":"2024-04-25T00:42:59.659638+0000","last_fullsized":"2024-04-25T00:42:59.659638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940935+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:42:59.940935+0000","last_peered":"2024-04-25T00:42:59.940935+0000","last_clean":"2024-04-25T00:42:59.940935+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:42:59.940935+0000","last_undegraded":"2024-04-25T00:42:59.940935+0000","last_fullsized":"2024-04-25T00:42:59.940935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014187+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:43:00.014187+0000","last_peered":"2024-04-25T00:43:00.014187+0000","last_clean":"2024-04-25T00:43:00.014187+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:43:00.014187+0000","last_undegraded":"2024-04-25T00:43:00.014187+0000","last_fullsized":"2024-04-25T00:43:00.014187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656790+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:42:59.656790+0000","last_peered":"2024-04-25T00:42:59.656790+0000","last_clean":"2024-04-25T00:42:59.656790+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:42:59.656790+0000","last_undegraded":"2024-04-25T00:42:59.656790+0000","last_fullsized":"2024-04-25T00:42:59.656790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013062+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:43:00.013062+0000","last_peered":"2024-04-25T00:43:00.013062+0000","last_clean":"2024-04-25T00:43:00.013062+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:43:00.013062+0000","last_undegraded":"2024-04-25T00:43:00.013062+0000","last_fullsized":"2024-04-25T00:43:00.013062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014577+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:43:00.014577+0000","last_peered":"2024-04-25T00:43:00.014577+0000","last_clean":"2024-04-25T00:43:00.014577+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:43:00.014577+0000","last_undegraded":"2024-04-25T00:43:00.014577+0000","last_fullsized":"2024-04-25T00:43:00.014577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658598+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:42:59.658598+0000","last_peered":"2024-04-25T00:42:59.658598+0000","last_clean":"2024-04-25T00:42:59.658598+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:42:59.658598+0000","last_undegraded":"2024-04-25T00:42:59.658598+0000","last_fullsized":"2024-04-25T00:42:59.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015445+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:43:00.015445+0000","last_peered":"2024-04-25T00:43:00.015445+0000","last_clean":"2024-04-25T00:43:00.015445+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:43:00.015445+0000","last_undegraded":"2024-04-25T00:43:00.015445+0000","last_fullsized":"2024-04-25T00:43:00.015445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662931+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:42:59.662931+0000","last_peered":"2024-04-25T00:42:59.662931+0000","last_clean":"2024-04-25T00:42:59.662931+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:42:59.662931+0000","last_undegraded":"2024-04-25T00:42:59.662931+0000","last_fullsized":"2024-04-25T00:42:59.662931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014447+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:43:00.014447+0000","last_peered":"2024-04-25T00:43:00.014447+0000","last_clean":"2024-04-25T00:43:00.014447+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:43:00.014447+0000","last_undegraded":"2024-04-25T00:43:00.014447+0000","last_fullsized":"2024-04-25T00:43:00.014447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013170+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:43:00.013170+0000","last_peered":"2024-04-25T00:43:00.013170+0000","last_clean":"2024-04-25T00:43:00.013170+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:43:00.013170+0000","last_undegraded":"2024-04-25T00:43:00.013170+0000","last_fullsized":"2024-04-25T00:43:00.013170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940601+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:42:59.940601+0000","last_peered":"2024-04-25T00:42:59.940601+0000","last_clean":"2024-04-25T00:42:59.940601+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:42:59.940601+0000","last_undegraded":"2024-04-25T00:42:59.940601+0000","last_fullsized":"2024-04-25T00:42:59.940601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012818+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:43:00.012818+0000","last_peered":"2024-04-25T00:43:00.012818+0000","last_clean":"2024-04-25T00:43:00.012818+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:43:00.012818+0000","last_undegraded":"2024-04-25T00:43:00.012818+0000","last_fullsized":"2024-04-25T00:43:00.012818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941753+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:42:59.941753+0000","last_peered":"2024-04-25T00:42:59.941753+0000","last_clean":"2024-04-25T00:42:59.941753+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:42:59.941753+0000","last_undegraded":"2024-04-25T00:42:59.941753+0000","last_fullsized":"2024-04-25T00:42:59.941753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663637+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:42:59.663637+0000","last_peered":"2024-04-25T00:42:59.663637+0000","last_clean":"2024-04-25T00:42:59.663637+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:42:59.663637+0000","last_undegraded":"2024-04-25T00:42:59.663637+0000","last_fullsized":"2024-04-25T00:42:59.663637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657292+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:42:59.657292+0000","last_peered":"2024-04-25T00:42:59.657292+0000","last_clean":"2024-04-25T00:42:59.657292+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:42:59.657292+0000","last_undegraded":"2024-04-25T00:42:59.657292+0000","last_fullsized":"2024-04-25T00:42:59.657292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941489+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:42:59.941489+0000","last_peered":"2024-04-25T00:42:59.941489+0000","last_clean":"2024-04-25T00:42:59.941489+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:42:59.941489+0000","last_undegraded":"2024-04-25T00:42:59.941489+0000","last_fullsized":"2024-04-25T00:42:59.941489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014644+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:43:00.014644+0000","last_peered":"2024-04-25T00:43:00.014644+0000","last_clean":"2024-04-25T00:43:00.014644+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:43:00.014644+0000","last_undegraded":"2024-04-25T00:43:00.014644+0000","last_fullsized":"2024-04-25T00:43:00.014644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013656+0000","last_change":"2024-04-25T00:42:58.693414+0000","last_active":"2024-04-25T00:43:00.013656+0000","last_peered":"2024-04-25T00:43:00.013656+0000","last_clean":"2024-04-25T00:43:00.013656+0000","last_became_active":"2024-04-25T00:42:58.693071+0000","last_became_peered":"2024-04-25T00:42:58.693071+0000","last_unstale":"2024-04-25T00:43:00.013656+0000","last_undegraded":"2024-04-25T00:43:00.013656+0000","last_fullsized":"2024-04-25T00:43:00.013656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658896+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:42:59.658896+0000","last_peered":"2024-04-25T00:42:59.658896+0000","last_clean":"2024-04-25T00:42:59.658896+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:42:59.658896+0000","last_undegraded":"2024-04-25T00:42:59.658896+0000","last_fullsized":"2024-04-25T00:42:59.658896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662637+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:42:59.662637+0000","last_peered":"2024-04-25T00:42:59.662637+0000","last_clean":"2024-04-25T00:42:59.662637+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:42:59.662637+0000","last_undegraded":"2024-04-25T00:42:59.662637+0000","last_fullsized":"2024-04-25T00:42:59.662637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658748+0000","last_change":"2024-04-25T00:42:58.689399+0000","last_active":"2024-04-25T00:42:59.658748+0000","last_peered":"2024-04-25T00:42:59.658748+0000","last_clean":"2024-04-25T00:42:59.658748+0000","last_became_active":"2024-04-25T00:42:58.689167+0000","last_became_peered":"2024-04-25T00:42:58.689167+0000","last_unstale":"2024-04-25T00:42:59.658748+0000","last_undegraded":"2024-04-25T00:42:59.658748+0000","last_fullsized":"2024-04-25T00:42:59.658748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:06.431173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941031+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:42:59.941031+0000","last_peered":"2024-04-25T00:42:59.941031+0000","last_clean":"2024-04-25T00:42:59.941031+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:42:59.941031+0000","last_undegraded":"2024-04-25T00:42:59.941031+0000","last_fullsized":"2024-04-25T00:42:59.941031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014851+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:43:00.014851+0000","last_peered":"2024-04-25T00:43:00.014851+0000","last_clean":"2024-04-25T00:43:00.014851+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:43:00.014851+0000","last_undegraded":"2024-04-25T00:43:00.014851+0000","last_fullsized":"2024-04-25T00:43:00.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662544+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:42:59.662544+0000","last_peered":"2024-04-25T00:42:59.662544+0000","last_clean":"2024-04-25T00:42:59.662544+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:42:59.662544+0000","last_undegraded":"2024-04-25T00:42:59.662544+0000","last_fullsized":"2024-04-25T00:42:59.662544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940666+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:42:59.940666+0000","last_peered":"2024-04-25T00:42:59.940666+0000","last_clean":"2024-04-25T00:42:59.940666+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:42:59.940666+0000","last_undegraded":"2024-04-25T00:42:59.940666+0000","last_fullsized":"2024-04-25T00:42:59.940666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940621+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:42:59.940621+0000","last_peered":"2024-04-25T00:42:59.940621+0000","last_clean":"2024-04-25T00:42:59.940621+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:42:59.940621+0000","last_undegraded":"2024-04-25T00:42:59.940621+0000","last_fullsized":"2024-04-25T00:42:59.940621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012893+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:43:00.012893+0000","last_peered":"2024-04-25T00:43:00.012893+0000","last_clean":"2024-04-25T00:43:00.012893+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:43:00.012893+0000","last_undegraded":"2024-04-25T00:43:00.012893+0000","last_fullsized":"2024-04-25T00:43:00.012893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656623+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:42:59.656623+0000","last_peered":"2024-04-25T00:42:59.656623+0000","last_clean":"2024-04-25T00:42:59.656623+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:42:59.656623+0000","last_undegraded":"2024-04-25T00:42:59.656623+0000","last_fullsized":"2024-04-25T00:42:59.656623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663244+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:42:59.663244+0000","last_peered":"2024-04-25T00:42:59.663244+0000","last_clean":"2024-04-25T00:42:59.663244+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:42:59.663244+0000","last_undegraded":"2024-04-25T00:42:59.663244+0000","last_fullsized":"2024-04-25T00:42:59.663244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013906+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:43:00.013906+0000","last_peered":"2024-04-25T00:43:00.013906+0000","last_clean":"2024-04-25T00:43:00.013906+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:43:00.013906+0000","last_undegraded":"2024-04-25T00:43:00.013906+0000","last_fullsized":"2024-04-25T00:43:00.013906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013989+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:43:00.013989+0000","last_peered":"2024-04-25T00:43:00.013989+0000","last_clean":"2024-04-25T00:43:00.013989+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:43:00.013989+0000","last_undegraded":"2024-04-25T00:43:00.013989+0000","last_fullsized":"2024-04-25T00:43:00.013989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014621+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:43:00.014621+0000","last_peered":"2024-04-25T00:43:00.014621+0000","last_clean":"2024-04-25T00:43:00.014621+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:43:00.014621+0000","last_undegraded":"2024-04-25T00:43:00.014621+0000","last_fullsized":"2024-04-25T00:43:00.014621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657481+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:42:59.657481+0000","last_peered":"2024-04-25T00:42:59.657481+0000","last_clean":"2024-04-25T00:42:59.657481+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:42:59.657481+0000","last_undegraded":"2024-04-25T00:42:59.657481+0000","last_fullsized":"2024-04-25T00:42:59.657481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659786+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:42:59.659786+0000","last_peered":"2024-04-25T00:42:59.659786+0000","last_clean":"2024-04-25T00:42:59.659786+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:42:59.659786+0000","last_undegraded":"2024-04-25T00:42:59.659786+0000","last_fullsized":"2024-04-25T00:42:59.659786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662829+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:42:59.662829+0000","last_peered":"2024-04-25T00:42:59.662829+0000","last_clean":"2024-04-25T00:42:59.662829+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:42:59.662829+0000","last_undegraded":"2024-04-25T00:42:59.662829+0000","last_fullsized":"2024-04-25T00:42:59.662829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015191+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:43:00.015191+0000","last_peered":"2024-04-25T00:43:00.015191+0000","last_clean":"2024-04-25T00:43:00.015191+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:43:00.015191+0000","last_undegraded":"2024-04-25T00:43:00.015191+0000","last_fullsized":"2024-04-25T00:43:00.015191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014789+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:43:00.014789+0000","last_peered":"2024-04-25T00:43:00.014789+0000","last_clean":"2024-04-25T00:43:00.014789+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:43:00.014789+0000","last_undegraded":"2024-04-25T00:43:00.014789+0000","last_fullsized":"2024-04-25T00:43:00.014789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014462+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:43:00.014462+0000","last_peered":"2024-04-25T00:43:00.014462+0000","last_clean":"2024-04-25T00:43:00.014462+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:43:00.014462+0000","last_undegraded":"2024-04-25T00:43:00.014462+0000","last_fullsized":"2024-04-25T00:43:00.014462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013395+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:43:00.013395+0000","last_peered":"2024-04-25T00:43:00.013395+0000","last_clean":"2024-04-25T00:43:00.013395+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:43:00.013395+0000","last_undegraded":"2024-04-25T00:43:00.013395+0000","last_fullsized":"2024-04-25T00:43:00.013395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014992+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:43:00.014992+0000","last_peered":"2024-04-25T00:43:00.014992+0000","last_clean":"2024-04-25T00:43:00.014992+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:43:00.014992+0000","last_undegraded":"2024-04-25T00:43:00.014992+0000","last_fullsized":"2024-04-25T00:43:00.014992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662429+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:42:59.662429+0000","last_peered":"2024-04-25T00:42:59.662429+0000","last_clean":"2024-04-25T00:42:59.662429+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:42:59.662429+0000","last_undegraded":"2024-04-25T00:42:59.662429+0000","last_fullsized":"2024-04-25T00:42:59.662429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662799+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:42:59.662799+0000","last_peered":"2024-04-25T00:42:59.662799+0000","last_clean":"2024-04-25T00:42:59.662799+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:42:59.662799+0000","last_undegraded":"2024-04-25T00:42:59.662799+0000","last_fullsized":"2024-04-25T00:42:59.662799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658133+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:42:59.658133+0000","last_peered":"2024-04-25T00:42:59.658133+0000","last_clean":"2024-04-25T00:42:59.658133+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:42:59.658133+0000","last_undegraded":"2024-04-25T00:42:59.658133+0000","last_fullsized":"2024-04-25T00:42:59.658133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014619+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:43:00.014619+0000","last_peered":"2024-04-25T00:43:00.014619+0000","last_clean":"2024-04-25T00:43:00.014619+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:43:00.014619+0000","last_undegraded":"2024-04-25T00:43:00.014619+0000","last_fullsized":"2024-04-25T00:43:00.014619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659249+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:42:59.659249+0000","last_peered":"2024-04-25T00:42:59.659249+0000","last_clean":"2024-04-25T00:42:59.659249+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:42:59.659249+0000","last_undegraded":"2024-04-25T00:42:59.659249+0000","last_fullsized":"2024-04-25T00:42:59.659249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015947+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:43:00.015947+0000","last_peered":"2024-04-25T00:43:00.015947+0000","last_clean":"2024-04-25T00:43:00.015947+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:43:00.015947+0000","last_undegraded":"2024-04-25T00:43:00.015947+0000","last_fullsized":"2024-04-25T00:43:00.015947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658685+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:42:59.658685+0000","last_peered":"2024-04-25T00:42:59.658685+0000","last_clean":"2024-04-25T00:42:59.658685+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:42:59.658685+0000","last_undegraded":"2024-04-25T00:42:59.658685+0000","last_fullsized":"2024-04-25T00:42:59.658685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016444+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:43:00.016444+0000","last_peered":"2024-04-25T00:43:00.016444+0000","last_clean":"2024-04-25T00:43:00.016444+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:43:00.016444+0000","last_undegraded":"2024-04-25T00:43:00.016444+0000","last_fullsized":"2024-04-25T00:43:00.016444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657468+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:42:59.657468+0000","last_peered":"2024-04-25T00:42:59.657468+0000","last_clean":"2024-04-25T00:42:59.657468+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:42:59.657468+0000","last_undegraded":"2024-04-25T00:42:59.657468+0000","last_fullsized":"2024-04-25T00:42:59.657468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013590+0000","last_change":"2024-04-25T00:42:58.693586+0000","last_active":"2024-04-25T00:43:00.013590+0000","last_peered":"2024-04-25T00:43:00.013590+0000","last_clean":"2024-04-25T00:43:00.013590+0000","last_became_active":"2024-04-25T00:42:58.692506+0000","last_became_peered":"2024-04-25T00:42:58.692506+0000","last_unstale":"2024-04-25T00:43:00.013590+0000","last_undegraded":"2024-04-25T00:43:00.013590+0000","last_fullsized":"2024-04-25T00:43:00.013590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658281+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:42:59.658281+0000","last_peered":"2024-04-25T00:42:59.658281+0000","last_clean":"2024-04-25T00:42:59.658281+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:42:59.658281+0000","last_undegraded":"2024-04-25T00:42:59.658281+0000","last_fullsized":"2024-04-25T00:42:59.658281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015234+0000","last_change":"2024-04-25T00:42:58.698024+0000","last_active":"2024-04-25T00:43:00.015234+0000","last_peered":"2024-04-25T00:43:00.015234+0000","last_clean":"2024-04-25T00:43:00.015234+0000","last_became_active":"2024-04-25T00:42:58.697733+0000","last_became_peered":"2024-04-25T00:42:58.697733+0000","last_unstale":"2024-04-25T00:43:00.015234+0000","last_undegraded":"2024-04-25T00:43:00.015234+0000","last_fullsized":"2024-04-25T00:43:00.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013464+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:00.013464+0000","last_peered":"2024-04-25T00:43:00.013464+0000","last_clean":"2024-04-25T00:43:00.013464+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:00.013464+0000","last_undegraded":"2024-04-25T00:43:00.013464+0000","last_fullsized":"2024-04-25T00:43:00.013464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013023+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:43:00.013023+0000","last_peered":"2024-04-25T00:43:00.013023+0000","last_clean":"2024-04-25T00:43:00.013023+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:43:00.013023+0000","last_undegraded":"2024-04-25T00:43:00.013023+0000","last_fullsized":"2024-04-25T00:43:00.013023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941039+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:42:59.941039+0000","last_peered":"2024-04-25T00:42:59.941039+0000","last_clean":"2024-04-25T00:42:59.941039+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:42:59.941039+0000","last_undegraded":"2024-04-25T00:42:59.941039+0000","last_fullsized":"2024-04-25T00:42:59.941039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013385+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:43:00.013385+0000","last_peered":"2024-04-25T00:43:00.013385+0000","last_clean":"2024-04-25T00:43:00.013385+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:43:00.013385+0000","last_undegraded":"2024-04-25T00:43:00.013385+0000","last_fullsized":"2024-04-25T00:43:00.013385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015080+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:43:00.015080+0000","last_peered":"2024-04-25T00:43:00.015080+0000","last_clean":"2024-04-25T00:43:00.015080+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:43:00.015080+0000","last_undegraded":"2024-04-25T00:43:00.015080+0000","last_fullsized":"2024-04-25T00:43:00.015080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658309+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:42:59.658309+0000","last_peered":"2024-04-25T00:42:59.658309+0000","last_clean":"2024-04-25T00:42:59.658309+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:42:59.658309+0000","last_undegraded":"2024-04-25T00:42:59.658309+0000","last_fullsized":"2024-04-25T00:42:59.658309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656914+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:42:59.656914+0000","last_peered":"2024-04-25T00:42:59.656914+0000","last_clean":"2024-04-25T00:42:59.656914+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:42:59.656914+0000","last_undegraded":"2024-04-25T00:42:59.656914+0000","last_fullsized":"2024-04-25T00:42:59.656914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013694+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:43:00.013694+0000","last_peered":"2024-04-25T00:43:00.013694+0000","last_clean":"2024-04-25T00:43:00.013694+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:43:00.013694+0000","last_undegraded":"2024-04-25T00:43:00.013694+0000","last_fullsized":"2024-04-25T00:43:00.013694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659040+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:42:59.659040+0000","last_peered":"2024-04-25T00:42:59.659040+0000","last_clean":"2024-04-25T00:42:59.659040+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:42:59.659040+0000","last_undegraded":"2024-04-25T00:42:59.659040+0000","last_fullsized":"2024-04-25T00:42:59.659040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657076+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:42:59.657076+0000","last_peered":"2024-04-25T00:42:59.657076+0000","last_clean":"2024-04-25T00:42:59.657076+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:42:59.657076+0000","last_undegraded":"2024-04-25T00:42:59.657076+0000","last_fullsized":"2024-04-25T00:42:59.657076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014402+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:43:00.014402+0000","last_peered":"2024-04-25T00:43:00.014402+0000","last_clean":"2024-04-25T00:43:00.014402+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:43:00.014402+0000","last_undegraded":"2024-04-25T00:43:00.014402+0000","last_fullsized":"2024-04-25T00:43:00.014402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015510+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:43:00.015510+0000","last_peered":"2024-04-25T00:43:00.015510+0000","last_clean":"2024-04-25T00:43:00.015510+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:43:00.015510+0000","last_undegraded":"2024-04-25T00:43:00.015510+0000","last_fullsized":"2024-04-25T00:43:00.015510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656937+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:42:59.656937+0000","last_peered":"2024-04-25T00:42:59.656937+0000","last_clean":"2024-04-25T00:42:59.656937+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:42:59.656937+0000","last_undegraded":"2024-04-25T00:42:59.656937+0000","last_fullsized":"2024-04-25T00:42:59.656937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657625+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:42:59.657625+0000","last_peered":"2024-04-25T00:42:59.657625+0000","last_clean":"2024-04-25T00:42:59.657625+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:42:59.657625+0000","last_undegraded":"2024-04-25T00:42:59.657625+0000","last_fullsized":"2024-04-25T00:42:59.657625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941751+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:42:59.941751+0000","last_peered":"2024-04-25T00:42:59.941751+0000","last_clean":"2024-04-25T00:42:59.941751+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:42:59.941751+0000","last_undegraded":"2024-04-25T00:42:59.941751+0000","last_fullsized":"2024-04-25T00:42:59.941751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663349+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:42:59.663349+0000","last_peered":"2024-04-25T00:42:59.663349+0000","last_clean":"2024-04-25T00:42:59.663349+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:42:59.663349+0000","last_undegraded":"2024-04-25T00:42:59.663349+0000","last_fullsized":"2024-04-25T00:42:59.663349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014387+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:43:00.014387+0000","last_peered":"2024-04-25T00:43:00.014387+0000","last_clean":"2024-04-25T00:43:00.014387+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:43:00.014387+0000","last_undegraded":"2024-04-25T00:43:00.014387+0000","last_fullsized":"2024-04-25T00:43:00.014387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013174+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:43:00.013174+0000","last_peered":"2024-04-25T00:43:00.013174+0000","last_clean":"2024-04-25T00:43:00.013174+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:43:00.013174+0000","last_undegraded":"2024-04-25T00:43:00.013174+0000","last_fullsized":"2024-04-25T00:43:00.013174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659268+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:42:59.659268+0000","last_peered":"2024-04-25T00:42:59.659268+0000","last_clean":"2024-04-25T00:42:59.659268+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:42:59.659268+0000","last_undegraded":"2024-04-25T00:42:59.659268+0000","last_fullsized":"2024-04-25T00:42:59.659268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656849+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:42:59.656849+0000","last_peered":"2024-04-25T00:42:59.656849+0000","last_clean":"2024-04-25T00:42:59.656849+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:42:59.656849+0000","last_undegraded":"2024-04-25T00:42:59.656849+0000","last_fullsized":"2024-04-25T00:42:59.656849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941104+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:42:59.941104+0000","last_peered":"2024-04-25T00:42:59.941104+0000","last_clean":"2024-04-25T00:42:59.941104+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:42:59.941104+0000","last_undegraded":"2024-04-25T00:42:59.941104+0000","last_fullsized":"2024-04-25T00:42:59.941104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657161+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:42:59.657161+0000","last_peered":"2024-04-25T00:42:59.657161+0000","last_clean":"2024-04-25T00:42:59.657161+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:42:59.657161+0000","last_undegraded":"2024-04-25T00:42:59.657161+0000","last_fullsized":"2024-04-25T00:42:59.657161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941098+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:42:59.941098+0000","last_peered":"2024-04-25T00:42:59.941098+0000","last_clean":"2024-04-25T00:42:59.941098+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:42:59.941098+0000","last_undegraded":"2024-04-25T00:42:59.941098+0000","last_fullsized":"2024-04-25T00:42:59.941098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015070+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:43:00.015070+0000","last_peered":"2024-04-25T00:43:00.015070+0000","last_clean":"2024-04-25T00:43:00.015070+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:43:00.015070+0000","last_undegraded":"2024-04-25T00:43:00.015070+0000","last_fullsized":"2024-04-25T00:43:00.015070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662834+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:42:59.662834+0000","last_peered":"2024-04-25T00:42:59.662834+0000","last_clean":"2024-04-25T00:42:59.662834+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:42:59.662834+0000","last_undegraded":"2024-04-25T00:42:59.662834+0000","last_fullsized":"2024-04-25T00:42:59.662834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014334+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:43:00.014334+0000","last_peered":"2024-04-25T00:43:00.014334+0000","last_clean":"2024-04-25T00:43:00.014334+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:43:00.014334+0000","last_undegraded":"2024-04-25T00:43:00.014334+0000","last_fullsized":"2024-04-25T00:43:00.014334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014890+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:43:00.014890+0000","last_peered":"2024-04-25T00:43:00.014890+0000","last_clean":"2024-04-25T00:43:00.014890+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:43:00.014890+0000","last_undegraded":"2024-04-25T00:43:00.014890+0000","last_fullsized":"2024-04-25T00:43:00.014890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013948+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:43:00.013948+0000","last_peered":"2024-04-25T00:43:00.013948+0000","last_clean":"2024-04-25T00:43:00.013948+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:43:00.013948+0000","last_undegraded":"2024-04-25T00:43:00.013948+0000","last_fullsized":"2024-04-25T00:43:00.013948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015334+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:43:00.015334+0000","last_peered":"2024-04-25T00:43:00.015334+0000","last_clean":"2024-04-25T00:43:00.015334+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:43:00.015334+0000","last_undegraded":"2024-04-25T00:43:00.015334+0000","last_fullsized":"2024-04-25T00:43:00.015334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014217+0000","last_change":"2024-04-25T00:42:58.700939+0000","last_active":"2024-04-25T00:43:00.014217+0000","last_peered":"2024-04-25T00:43:00.014217+0000","last_clean":"2024-04-25T00:43:00.014217+0000","last_became_active":"2024-04-25T00:42:58.700177+0000","last_became_peered":"2024-04-25T00:42:58.700177+0000","last_unstale":"2024-04-25T00:43:00.014217+0000","last_undegraded":"2024-04-25T00:43:00.014217+0000","last_fullsized":"2024-04-25T00:43:00.014217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:36.086387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941542+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:42:59.941542+0000","last_peered":"2024-04-25T00:42:59.941542+0000","last_clean":"2024-04-25T00:42:59.941542+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:42:59.941542+0000","last_undegraded":"2024-04-25T00:42:59.941542+0000","last_fullsized":"2024-04-25T00:42:59.941542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941612+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:42:59.941612+0000","last_peered":"2024-04-25T00:42:59.941612+0000","last_clean":"2024-04-25T00:42:59.941612+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:42:59.941612+0000","last_undegraded":"2024-04-25T00:42:59.941612+0000","last_fullsized":"2024-04-25T00:42:59.941612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015091+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:43:00.015091+0000","last_peered":"2024-04-25T00:43:00.015091+0000","last_clean":"2024-04-25T00:43:00.015091+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:43:00.015091+0000","last_undegraded":"2024-04-25T00:43:00.015091+0000","last_fullsized":"2024-04-25T00:43:00.015091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:43:02.719 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:43:02.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 Processor -- start 2024-04-25T00:43:02.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 -- start start 2024-04-25T00:43:02.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9f80cf270 0x7fd9f80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd9f8147f20 0x7fd9f8070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f806b4b0 con 0x7fd9f8147b40 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f806b770 con 0x7fd9f80cf270 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9fe31c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f806ba50 con 0x7fd9f8147f20 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9f77fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9f80cf270 0x7fd9f80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9f77fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9f80cf270 0x7fd9f80706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52912/0 (socket says 172.21.15.129:52912) 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.872+0000 7fd9f77fe640 1 -- 172.21.15.129:0/3622109475 learned_addr learned my addr 172.21.15.129:0/3622109475 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fc8b9640 1 --2- 172.21.15.129:0/3622109475 >> v2:172.21.15.196:3300/0 conn(0x7fd9f8147f20 0x7fd9f8070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f77fe640 1 -- 172.21.15.129:0/3622109475 >> v2:172.21.15.196:3300/0 conn(0x7fd9f8147f20 msgr2=0x7fd9f8070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f77fe640 1 --2- 172.21.15.129:0/3622109475 >> v2:172.21.15.196:3300/0 conn(0x7fd9f8147f20 0x7fd9f8070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f77fe640 1 -- 172.21.15.129:0/3622109475 >> v1:172.21.15.196:6789/0 conn(0x7fd9f8147b40 legacy=0x7fd9f80cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f77fe640 1 -- 172.21.15.129:0/3622109475 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f806bd30 con 0x7fd9f80cf270 2024-04-25T00:43:02.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fc8b9640 1 --2- 172.21.15.129:0/3622109475 >> v2:172.21.15.196:3300/0 conn(0x7fd9f8147f20 0x7fd9f8070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:02.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f77fe640 1 --2- 172.21.15.129:0/3622109475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f80706d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80cda80 tx=0x7fd9ec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2770b05e11930f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f6ffd640 1 -- 172.21.15.129:0/3622109475 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0b2070 con 0x7fd9f80cf270 2024-04-25T00:43:02.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f6ffd640 1 -- 172.21.15.129:0/3622109475 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0a9070 con 0x7fd9f80cf270 2024-04-25T00:43:02.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9f6ffd640 1 -- 172.21.15.129:0/3622109475 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0ad880 con 0x7fd9f80cf270 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3622109475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 msgr2=0x7fd9f80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3622109475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f80706d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80cda80 tx=0x7fd9ec01c450 comp rx=0 tx=0).stop 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3622109475 shutdown_connections 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3622109475 >> v2:172.21.15.196:3300/0 conn(0x7fd9f8147f20 0x7fd9f8070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3622109475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f80706d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3622109475 >> 172.21.15.129:0/3622109475 conn(0x7fd9f80b2380 msgr2=0x7fd9f80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3622109475 shutdown_connections 2024-04-25T00:43:02.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.876+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3622109475 wait complete. 2024-04-25T00:43:02.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 Processor -- start 2024-04-25T00:43:02.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- start start 2024-04-25T00:43:02.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f813c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9f8147b40 0x7fd9f813cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9f8147f20 0x7fd9f813d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80ccac0 con 0x7fd9f80cf270 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80cc940 con 0x7fd9f8147f20 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f80ccc40 con 0x7fd9f8147b40 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f813c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f813c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52918/0 (socket says 172.21.15.129:52918) 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 -- 172.21.15.129:0/3470852596 learned_addr learned my addr 172.21.15.129:0/3470852596 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f77fe640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9f8147b40 0x7fd9f813cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fc8b9640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9f8147f20 0x7fd9f813d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 -- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9f8147b40 msgr2=0x7fd9f813cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9f8147b40 0x7fd9f813cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 -- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9f8147f20 msgr2=0x7fd9f813d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9f8147f20 0x7fd9f813d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 -- 172.21.15.129:0/3470852596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f8146560 con 0x7fd9f80cf270 2024-04-25T00:43:02.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f77fe640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9f8147b40 0x7fd9f813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:02.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f7fff640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f813c620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c450 tx=0x7fd9e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8012830 con 0x7fd9f80cf270 2024-04-25T00:43:02.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e8017070 con 0x7fd9f80cf270 2024-04-25T00:43:02.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f8146840 con 0x7fd9f80cf270 2024-04-25T00:43:02.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8012c30 con 0x7fd9f80cf270 2024-04-25T00:43:02.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f8146e00 con 0x7fd9f80cf270 2024-04-25T00:43:02.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f8147040 con 0x7fd9f80cf270 2024-04-25T00:43:02.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9e801c460 con 0x7fd9f80cf270 2024-04-25T00:43:02.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f4ff9640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9e40678c0 0x7fd9e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:02.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9e8085ac0 con 0x7fd9f80cf270 2024-04-25T00:43:02.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.880+0000 7fd9f77fe640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9e40678c0 0x7fd9e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:02.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.884+0000 7fd9f77fe640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9e40678c0 0x7fd9e4069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0af000 tx=0x7fd9ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:02.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:02.884+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd9e8056600 con 0x7fd9f80cf270 2024-04-25T00:43:03.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.032+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9f806c630 con 0x7fd9e40678c0 2024-04-25T00:43:03.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.052+0000 7fd9f4ff9640 1 -- 172.21.15.129:0/3470852596 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fd9f806c630 con 0x7fd9e40678c0 2024-04-25T00:43:03.058 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:03.059 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:43:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9e40678c0 msgr2=0x7fd9e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9e40678c0 0x7fd9e4069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0af000 tx=0x7fd9ec008040 comp rx=0 tx=0).stop 2024-04-25T00:43:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 msgr2=0x7fd9f813c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f813c620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c450 tx=0x7fd9e800c910 comp rx=0 tx=0).stop 2024-04-25T00:43:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 shutdown_connections 2024-04-25T00:43:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9e40678c0 0x7fd9e4069d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9f8147b40 0x7fd9f813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9f8147f20 0x7fd9f813d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 --2- 172.21.15.129:0/3470852596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9f80cf270 0x7fd9f813c620 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 >> 172.21.15.129:0/3470852596 conn(0x7fd9f80b2380 msgr2=0x7fd9f8072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 shutdown_connections 2024-04-25T00:43:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.056+0000 7fd9fe31c640 1 -- 172.21.15.129:0/3470852596 wait complete. 2024-04-25T00:43:03.074 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T00:43:02.605779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.980665"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016159+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:43:00.016159+0000","last_peered":"2024-04-25T00:43:00.016159+0000","last_clean":"2024-04-25T00:43:00.016159+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:43:00.016159+0000","last_undegraded":"2024-04-25T00:43:00.016159+0000","last_fullsized":"2024-04-25T00:43:00.016159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657681+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:42:59.657681+0000","last_peered":"2024-04-25T00:42:59.657681+0000","last_clean":"2024-04-25T00:42:59.657681+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:42:59.657681+0000","last_undegraded":"2024-04-25T00:42:59.657681+0000","last_fullsized":"2024-04-25T00:42:59.657681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014295+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:43:00.014295+0000","last_peered":"2024-04-25T00:43:00.014295+0000","last_clean":"2024-04-25T00:43:00.014295+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:43:00.014295+0000","last_undegraded":"2024-04-25T00:43:00.014295+0000","last_fullsized":"2024-04-25T00:43:00.014295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013425+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:43:00.013425+0000","last_peered":"2024-04-25T00:43:00.013425+0000","last_clean":"2024-04-25T00:43:00.013425+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:43:00.013425+0000","last_undegraded":"2024-04-25T00:43:00.013425+0000","last_fullsized":"2024-04-25T00:43:00.013425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014816+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:43:00.014816+0000","last_peered":"2024-04-25T00:43:00.014816+0000","last_clean":"2024-04-25T00:43:00.014816+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:43:00.014816+0000","last_undegraded":"2024-04-25T00:43:00.014816+0000","last_fullsized":"2024-04-25T00:43:00.014816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013220+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:43:00.013220+0000","last_peered":"2024-04-25T00:43:00.013220+0000","last_clean":"2024-04-25T00:43:00.013220+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:43:00.013220+0000","last_undegraded":"2024-04-25T00:43:00.013220+0000","last_fullsized":"2024-04-25T00:43:00.013220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013356+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:43:00.013356+0000","last_peered":"2024-04-25T00:43:00.013356+0000","last_clean":"2024-04-25T00:43:00.013356+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:43:00.013356+0000","last_undegraded":"2024-04-25T00:43:00.013356+0000","last_fullsized":"2024-04-25T00:43:00.013356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940441+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:42:59.940441+0000","last_peered":"2024-04-25T00:42:59.940441+0000","last_clean":"2024-04-25T00:42:59.940441+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:42:59.940441+0000","last_undegraded":"2024-04-25T00:42:59.940441+0000","last_fullsized":"2024-04-25T00:42:59.940441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016200+0000","last_change":"2024-04-25T00:42:58.698697+0000","last_active":"2024-04-25T00:43:00.016200+0000","last_peered":"2024-04-25T00:43:00.016200+0000","last_clean":"2024-04-25T00:43:00.016200+0000","last_became_active":"2024-04-25T00:42:58.698442+0000","last_became_peered":"2024-04-25T00:42:58.698442+0000","last_unstale":"2024-04-25T00:43:00.016200+0000","last_undegraded":"2024-04-25T00:43:00.016200+0000","last_fullsized":"2024-04-25T00:43:00.016200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663628+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:42:59.663628+0000","last_peered":"2024-04-25T00:42:59.663628+0000","last_clean":"2024-04-25T00:42:59.663628+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:42:59.663628+0000","last_undegraded":"2024-04-25T00:42:59.663628+0000","last_fullsized":"2024-04-25T00:42:59.663628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663300+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:42:59.663300+0000","last_peered":"2024-04-25T00:42:59.663300+0000","last_clean":"2024-04-25T00:42:59.663300+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:42:59.663300+0000","last_undegraded":"2024-04-25T00:42:59.663300+0000","last_fullsized":"2024-04-25T00:42:59.663300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659638+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:42:59.659638+0000","last_peered":"2024-04-25T00:42:59.659638+0000","last_clean":"2024-04-25T00:42:59.659638+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:42:59.659638+0000","last_undegraded":"2024-04-25T00:42:59.659638+0000","last_fullsized":"2024-04-25T00:42:59.659638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940935+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:42:59.940935+0000","last_peered":"2024-04-25T00:42:59.940935+0000","last_clean":"2024-04-25T00:42:59.940935+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:42:59.940935+0000","last_undegraded":"2024-04-25T00:42:59.940935+0000","last_fullsized":"2024-04-25T00:42:59.940935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014187+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:43:00.014187+0000","last_peered":"2024-04-25T00:43:00.014187+0000","last_clean":"2024-04-25T00:43:00.014187+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:43:00.014187+0000","last_undegraded":"2024-04-25T00:43:00.014187+0000","last_fullsized":"2024-04-25T00:43:00.014187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656790+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:42:59.656790+0000","last_peered":"2024-04-25T00:42:59.656790+0000","last_clean":"2024-04-25T00:42:59.656790+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:42:59.656790+0000","last_undegraded":"2024-04-25T00:42:59.656790+0000","last_fullsized":"2024-04-25T00:42:59.656790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013062+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:43:00.013062+0000","last_peered":"2024-04-25T00:43:00.013062+0000","last_clean":"2024-04-25T00:43:00.013062+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:43:00.013062+0000","last_undegraded":"2024-04-25T00:43:00.013062+0000","last_fullsized":"2024-04-25T00:43:00.013062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014577+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:43:00.014577+0000","last_peered":"2024-04-25T00:43:00.014577+0000","last_clean":"2024-04-25T00:43:00.014577+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:43:00.014577+0000","last_undegraded":"2024-04-25T00:43:00.014577+0000","last_fullsized":"2024-04-25T00:43:00.014577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658598+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:42:59.658598+0000","last_peered":"2024-04-25T00:42:59.658598+0000","last_clean":"2024-04-25T00:42:59.658598+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:42:59.658598+0000","last_undegraded":"2024-04-25T00:42:59.658598+0000","last_fullsized":"2024-04-25T00:42:59.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015445+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:43:00.015445+0000","last_peered":"2024-04-25T00:43:00.015445+0000","last_clean":"2024-04-25T00:43:00.015445+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:43:00.015445+0000","last_undegraded":"2024-04-25T00:43:00.015445+0000","last_fullsized":"2024-04-25T00:43:00.015445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662931+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:42:59.662931+0000","last_peered":"2024-04-25T00:42:59.662931+0000","last_clean":"2024-04-25T00:42:59.662931+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:42:59.662931+0000","last_undegraded":"2024-04-25T00:42:59.662931+0000","last_fullsized":"2024-04-25T00:42:59.662931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014447+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:43:00.014447+0000","last_peered":"2024-04-25T00:43:00.014447+0000","last_clean":"2024-04-25T00:43:00.014447+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:43:00.014447+0000","last_undegraded":"2024-04-25T00:43:00.014447+0000","last_fullsized":"2024-04-25T00:43:00.014447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013170+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:43:00.013170+0000","last_peered":"2024-04-25T00:43:00.013170+0000","last_clean":"2024-04-25T00:43:00.013170+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:43:00.013170+0000","last_undegraded":"2024-04-25T00:43:00.013170+0000","last_fullsized":"2024-04-25T00:43:00.013170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940601+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:42:59.940601+0000","last_peered":"2024-04-25T00:42:59.940601+0000","last_clean":"2024-04-25T00:42:59.940601+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:42:59.940601+0000","last_undegraded":"2024-04-25T00:42:59.940601+0000","last_fullsized":"2024-04-25T00:42:59.940601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012818+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:43:00.012818+0000","last_peered":"2024-04-25T00:43:00.012818+0000","last_clean":"2024-04-25T00:43:00.012818+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:43:00.012818+0000","last_undegraded":"2024-04-25T00:43:00.012818+0000","last_fullsized":"2024-04-25T00:43:00.012818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941753+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:42:59.941753+0000","last_peered":"2024-04-25T00:42:59.941753+0000","last_clean":"2024-04-25T00:42:59.941753+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:42:59.941753+0000","last_undegraded":"2024-04-25T00:42:59.941753+0000","last_fullsized":"2024-04-25T00:42:59.941753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663637+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:42:59.663637+0000","last_peered":"2024-04-25T00:42:59.663637+0000","last_clean":"2024-04-25T00:42:59.663637+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:42:59.663637+0000","last_undegraded":"2024-04-25T00:42:59.663637+0000","last_fullsized":"2024-04-25T00:42:59.663637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657292+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:42:59.657292+0000","last_peered":"2024-04-25T00:42:59.657292+0000","last_clean":"2024-04-25T00:42:59.657292+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:42:59.657292+0000","last_undegraded":"2024-04-25T00:42:59.657292+0000","last_fullsized":"2024-04-25T00:42:59.657292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941489+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:42:59.941489+0000","last_peered":"2024-04-25T00:42:59.941489+0000","last_clean":"2024-04-25T00:42:59.941489+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:42:59.941489+0000","last_undegraded":"2024-04-25T00:42:59.941489+0000","last_fullsized":"2024-04-25T00:42:59.941489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014644+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:43:00.014644+0000","last_peered":"2024-04-25T00:43:00.014644+0000","last_clean":"2024-04-25T00:43:00.014644+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:43:00.014644+0000","last_undegraded":"2024-04-25T00:43:00.014644+0000","last_fullsized":"2024-04-25T00:43:00.014644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013656+0000","last_change":"2024-04-25T00:42:58.693414+0000","last_active":"2024-04-25T00:43:00.013656+0000","last_peered":"2024-04-25T00:43:00.013656+0000","last_clean":"2024-04-25T00:43:00.013656+0000","last_became_active":"2024-04-25T00:42:58.693071+0000","last_became_peered":"2024-04-25T00:42:58.693071+0000","last_unstale":"2024-04-25T00:43:00.013656+0000","last_undegraded":"2024-04-25T00:43:00.013656+0000","last_fullsized":"2024-04-25T00:43:00.013656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658896+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:42:59.658896+0000","last_peered":"2024-04-25T00:42:59.658896+0000","last_clean":"2024-04-25T00:42:59.658896+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:42:59.658896+0000","last_undegraded":"2024-04-25T00:42:59.658896+0000","last_fullsized":"2024-04-25T00:42:59.658896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662637+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:42:59.662637+0000","last_peered":"2024-04-25T00:42:59.662637+0000","last_clean":"2024-04-25T00:42:59.662637+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:42:59.662637+0000","last_undegraded":"2024-04-25T00:42:59.662637+0000","last_fullsized":"2024-04-25T00:42:59.662637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658748+0000","last_change":"2024-04-25T00:42:58.689399+0000","last_active":"2024-04-25T00:42:59.658748+0000","last_peered":"2024-04-25T00:42:59.658748+0000","last_clean":"2024-04-25T00:42:59.658748+0000","last_became_active":"2024-04-25T00:42:58.689167+0000","last_became_peered":"2024-04-25T00:42:58.689167+0000","last_unstale":"2024-04-25T00:42:59.658748+0000","last_undegraded":"2024-04-25T00:42:59.658748+0000","last_fullsized":"2024-04-25T00:42:59.658748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:06.431173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941031+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:42:59.941031+0000","last_peered":"2024-04-25T00:42:59.941031+0000","last_clean":"2024-04-25T00:42:59.941031+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:42:59.941031+0000","last_undegraded":"2024-04-25T00:42:59.941031+0000","last_fullsized":"2024-04-25T00:42:59.941031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014851+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:43:00.014851+0000","last_peered":"2024-04-25T00:43:00.014851+0000","last_clean":"2024-04-25T00:43:00.014851+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:43:00.014851+0000","last_undegraded":"2024-04-25T00:43:00.014851+0000","last_fullsized":"2024-04-25T00:43:00.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662544+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:42:59.662544+0000","last_peered":"2024-04-25T00:42:59.662544+0000","last_clean":"2024-04-25T00:42:59.662544+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:42:59.662544+0000","last_undegraded":"2024-04-25T00:42:59.662544+0000","last_fullsized":"2024-04-25T00:42:59.662544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940666+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:42:59.940666+0000","last_peered":"2024-04-25T00:42:59.940666+0000","last_clean":"2024-04-25T00:42:59.940666+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:42:59.940666+0000","last_undegraded":"2024-04-25T00:42:59.940666+0000","last_fullsized":"2024-04-25T00:42:59.940666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940621+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:42:59.940621+0000","last_peered":"2024-04-25T00:42:59.940621+0000","last_clean":"2024-04-25T00:42:59.940621+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:42:59.940621+0000","last_undegraded":"2024-04-25T00:42:59.940621+0000","last_fullsized":"2024-04-25T00:42:59.940621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012893+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:43:00.012893+0000","last_peered":"2024-04-25T00:43:00.012893+0000","last_clean":"2024-04-25T00:43:00.012893+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:43:00.012893+0000","last_undegraded":"2024-04-25T00:43:00.012893+0000","last_fullsized":"2024-04-25T00:43:00.012893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656623+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:42:59.656623+0000","last_peered":"2024-04-25T00:42:59.656623+0000","last_clean":"2024-04-25T00:42:59.656623+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:42:59.656623+0000","last_undegraded":"2024-04-25T00:42:59.656623+0000","last_fullsized":"2024-04-25T00:42:59.656623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663244+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:42:59.663244+0000","last_peered":"2024-04-25T00:42:59.663244+0000","last_clean":"2024-04-25T00:42:59.663244+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:42:59.663244+0000","last_undegraded":"2024-04-25T00:42:59.663244+0000","last_fullsized":"2024-04-25T00:42:59.663244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013906+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:43:00.013906+0000","last_peered":"2024-04-25T00:43:00.013906+0000","last_clean":"2024-04-25T00:43:00.013906+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:43:00.013906+0000","last_undegraded":"2024-04-25T00:43:00.013906+0000","last_fullsized":"2024-04-25T00:43:00.013906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013989+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:43:00.013989+0000","last_peered":"2024-04-25T00:43:00.013989+0000","last_clean":"2024-04-25T00:43:00.013989+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:43:00.013989+0000","last_undegraded":"2024-04-25T00:43:00.013989+0000","last_fullsized":"2024-04-25T00:43:00.013989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014621+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:43:00.014621+0000","last_peered":"2024-04-25T00:43:00.014621+0000","last_clean":"2024-04-25T00:43:00.014621+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:43:00.014621+0000","last_undegraded":"2024-04-25T00:43:00.014621+0000","last_fullsized":"2024-04-25T00:43:00.014621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657481+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:42:59.657481+0000","last_peered":"2024-04-25T00:42:59.657481+0000","last_clean":"2024-04-25T00:42:59.657481+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:42:59.657481+0000","last_undegraded":"2024-04-25T00:42:59.657481+0000","last_fullsized":"2024-04-25T00:42:59.657481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659786+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:42:59.659786+0000","last_peered":"2024-04-25T00:42:59.659786+0000","last_clean":"2024-04-25T00:42:59.659786+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:42:59.659786+0000","last_undegraded":"2024-04-25T00:42:59.659786+0000","last_fullsized":"2024-04-25T00:42:59.659786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662829+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:42:59.662829+0000","last_peered":"2024-04-25T00:42:59.662829+0000","last_clean":"2024-04-25T00:42:59.662829+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:42:59.662829+0000","last_undegraded":"2024-04-25T00:42:59.662829+0000","last_fullsized":"2024-04-25T00:42:59.662829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015191+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:43:00.015191+0000","last_peered":"2024-04-25T00:43:00.015191+0000","last_clean":"2024-04-25T00:43:00.015191+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:43:00.015191+0000","last_undegraded":"2024-04-25T00:43:00.015191+0000","last_fullsized":"2024-04-25T00:43:00.015191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014789+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:43:00.014789+0000","last_peered":"2024-04-25T00:43:00.014789+0000","last_clean":"2024-04-25T00:43:00.014789+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:43:00.014789+0000","last_undegraded":"2024-04-25T00:43:00.014789+0000","last_fullsized":"2024-04-25T00:43:00.014789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014462+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:43:00.014462+0000","last_peered":"2024-04-25T00:43:00.014462+0000","last_clean":"2024-04-25T00:43:00.014462+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:43:00.014462+0000","last_undegraded":"2024-04-25T00:43:00.014462+0000","last_fullsized":"2024-04-25T00:43:00.014462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013395+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:43:00.013395+0000","last_peered":"2024-04-25T00:43:00.013395+0000","last_clean":"2024-04-25T00:43:00.013395+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:43:00.013395+0000","last_undegraded":"2024-04-25T00:43:00.013395+0000","last_fullsized":"2024-04-25T00:43:00.013395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014992+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:43:00.014992+0000","last_peered":"2024-04-25T00:43:00.014992+0000","last_clean":"2024-04-25T00:43:00.014992+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:43:00.014992+0000","last_undegraded":"2024-04-25T00:43:00.014992+0000","last_fullsized":"2024-04-25T00:43:00.014992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662429+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:42:59.662429+0000","last_peered":"2024-04-25T00:42:59.662429+0000","last_clean":"2024-04-25T00:42:59.662429+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:42:59.662429+0000","last_undegraded":"2024-04-25T00:42:59.662429+0000","last_fullsized":"2024-04-25T00:42:59.662429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662799+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:42:59.662799+0000","last_peered":"2024-04-25T00:42:59.662799+0000","last_clean":"2024-04-25T00:42:59.662799+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:42:59.662799+0000","last_undegraded":"2024-04-25T00:42:59.662799+0000","last_fullsized":"2024-04-25T00:42:59.662799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658133+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:42:59.658133+0000","last_peered":"2024-04-25T00:42:59.658133+0000","last_clean":"2024-04-25T00:42:59.658133+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:42:59.658133+0000","last_undegraded":"2024-04-25T00:42:59.658133+0000","last_fullsized":"2024-04-25T00:42:59.658133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014619+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:43:00.014619+0000","last_peered":"2024-04-25T00:43:00.014619+0000","last_clean":"2024-04-25T00:43:00.014619+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:43:00.014619+0000","last_undegraded":"2024-04-25T00:43:00.014619+0000","last_fullsized":"2024-04-25T00:43:00.014619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659249+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:42:59.659249+0000","last_peered":"2024-04-25T00:42:59.659249+0000","last_clean":"2024-04-25T00:42:59.659249+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:42:59.659249+0000","last_undegraded":"2024-04-25T00:42:59.659249+0000","last_fullsized":"2024-04-25T00:42:59.659249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015947+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:43:00.015947+0000","last_peered":"2024-04-25T00:43:00.015947+0000","last_clean":"2024-04-25T00:43:00.015947+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:43:00.015947+0000","last_undegraded":"2024-04-25T00:43:00.015947+0000","last_fullsized":"2024-04-25T00:43:00.015947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658685+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:42:59.658685+0000","last_peered":"2024-04-25T00:42:59.658685+0000","last_clean":"2024-04-25T00:42:59.658685+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:42:59.658685+0000","last_undegraded":"2024-04-25T00:42:59.658685+0000","last_fullsized":"2024-04-25T00:42:59.658685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016444+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:43:00.016444+0000","last_peered":"2024-04-25T00:43:00.016444+0000","last_clean":"2024-04-25T00:43:00.016444+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:43:00.016444+0000","last_undegraded":"2024-04-25T00:43:00.016444+0000","last_fullsized":"2024-04-25T00:43:00.016444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657468+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:42:59.657468+0000","last_peered":"2024-04-25T00:42:59.657468+0000","last_clean":"2024-04-25T00:42:59.657468+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:42:59.657468+0000","last_undegraded":"2024-04-25T00:42:59.657468+0000","last_fullsized":"2024-04-25T00:42:59.657468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013590+0000","last_change":"2024-04-25T00:42:58.693586+0000","last_active":"2024-04-25T00:43:00.013590+0000","last_peered":"2024-04-25T00:43:00.013590+0000","last_clean":"2024-04-25T00:43:00.013590+0000","last_became_active":"2024-04-25T00:42:58.692506+0000","last_became_peered":"2024-04-25T00:42:58.692506+0000","last_unstale":"2024-04-25T00:43:00.013590+0000","last_undegraded":"2024-04-25T00:43:00.013590+0000","last_fullsized":"2024-04-25T00:43:00.013590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658281+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:42:59.658281+0000","last_peered":"2024-04-25T00:42:59.658281+0000","last_clean":"2024-04-25T00:42:59.658281+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:42:59.658281+0000","last_undegraded":"2024-04-25T00:42:59.658281+0000","last_fullsized":"2024-04-25T00:42:59.658281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015234+0000","last_change":"2024-04-25T00:42:58.698024+0000","last_active":"2024-04-25T00:43:00.015234+0000","last_peered":"2024-04-25T00:43:00.015234+0000","last_clean":"2024-04-25T00:43:00.015234+0000","last_became_active":"2024-04-25T00:42:58.697733+0000","last_became_peered":"2024-04-25T00:42:58.697733+0000","last_unstale":"2024-04-25T00:43:00.015234+0000","last_undegraded":"2024-04-25T00:43:00.015234+0000","last_fullsized":"2024-04-25T00:43:00.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013464+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:00.013464+0000","last_peered":"2024-04-25T00:43:00.013464+0000","last_clean":"2024-04-25T00:43:00.013464+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:00.013464+0000","last_undegraded":"2024-04-25T00:43:00.013464+0000","last_fullsized":"2024-04-25T00:43:00.013464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013023+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:43:00.013023+0000","last_peered":"2024-04-25T00:43:00.013023+0000","last_clean":"2024-04-25T00:43:00.013023+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:43:00.013023+0000","last_undegraded":"2024-04-25T00:43:00.013023+0000","last_fullsized":"2024-04-25T00:43:00.013023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941039+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:42:59.941039+0000","last_peered":"2024-04-25T00:42:59.941039+0000","last_clean":"2024-04-25T00:42:59.941039+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:42:59.941039+0000","last_undegraded":"2024-04-25T00:42:59.941039+0000","last_fullsized":"2024-04-25T00:42:59.941039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013385+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:43:00.013385+0000","last_peered":"2024-04-25T00:43:00.013385+0000","last_clean":"2024-04-25T00:43:00.013385+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:43:00.013385+0000","last_undegraded":"2024-04-25T00:43:00.013385+0000","last_fullsized":"2024-04-25T00:43:00.013385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015080+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:43:00.015080+0000","last_peered":"2024-04-25T00:43:00.015080+0000","last_clean":"2024-04-25T00:43:00.015080+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:43:00.015080+0000","last_undegraded":"2024-04-25T00:43:00.015080+0000","last_fullsized":"2024-04-25T00:43:00.015080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658309+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:42:59.658309+0000","last_peered":"2024-04-25T00:42:59.658309+0000","last_clean":"2024-04-25T00:42:59.658309+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:42:59.658309+0000","last_undegraded":"2024-04-25T00:42:59.658309+0000","last_fullsized":"2024-04-25T00:42:59.658309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656914+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:42:59.656914+0000","last_peered":"2024-04-25T00:42:59.656914+0000","last_clean":"2024-04-25T00:42:59.656914+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:42:59.656914+0000","last_undegraded":"2024-04-25T00:42:59.656914+0000","last_fullsized":"2024-04-25T00:42:59.656914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013694+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:43:00.013694+0000","last_peered":"2024-04-25T00:43:00.013694+0000","last_clean":"2024-04-25T00:43:00.013694+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:43:00.013694+0000","last_undegraded":"2024-04-25T00:43:00.013694+0000","last_fullsized":"2024-04-25T00:43:00.013694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659040+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:42:59.659040+0000","last_peered":"2024-04-25T00:42:59.659040+0000","last_clean":"2024-04-25T00:42:59.659040+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:42:59.659040+0000","last_undegraded":"2024-04-25T00:42:59.659040+0000","last_fullsized":"2024-04-25T00:42:59.659040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657076+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:42:59.657076+0000","last_peered":"2024-04-25T00:42:59.657076+0000","last_clean":"2024-04-25T00:42:59.657076+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:42:59.657076+0000","last_undegraded":"2024-04-25T00:42:59.657076+0000","last_fullsized":"2024-04-25T00:42:59.657076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014402+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:43:00.014402+0000","last_peered":"2024-04-25T00:43:00.014402+0000","last_clean":"2024-04-25T00:43:00.014402+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:43:00.014402+0000","last_undegraded":"2024-04-25T00:43:00.014402+0000","last_fullsized":"2024-04-25T00:43:00.014402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015510+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:43:00.015510+0000","last_peered":"2024-04-25T00:43:00.015510+0000","last_clean":"2024-04-25T00:43:00.015510+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:43:00.015510+0000","last_undegraded":"2024-04-25T00:43:00.015510+0000","last_fullsized":"2024-04-25T00:43:00.015510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656937+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:42:59.656937+0000","last_peered":"2024-04-25T00:42:59.656937+0000","last_clean":"2024-04-25T00:42:59.656937+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:42:59.656937+0000","last_undegraded":"2024-04-25T00:42:59.656937+0000","last_fullsized":"2024-04-25T00:42:59.656937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657625+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:42:59.657625+0000","last_peered":"2024-04-25T00:42:59.657625+0000","last_clean":"2024-04-25T00:42:59.657625+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:42:59.657625+0000","last_undegraded":"2024-04-25T00:42:59.657625+0000","last_fullsized":"2024-04-25T00:42:59.657625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941751+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:42:59.941751+0000","last_peered":"2024-04-25T00:42:59.941751+0000","last_clean":"2024-04-25T00:42:59.941751+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:42:59.941751+0000","last_undegraded":"2024-04-25T00:42:59.941751+0000","last_fullsized":"2024-04-25T00:42:59.941751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663349+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:42:59.663349+0000","last_peered":"2024-04-25T00:42:59.663349+0000","last_clean":"2024-04-25T00:42:59.663349+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:42:59.663349+0000","last_undegraded":"2024-04-25T00:42:59.663349+0000","last_fullsized":"2024-04-25T00:42:59.663349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014387+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:43:00.014387+0000","last_peered":"2024-04-25T00:43:00.014387+0000","last_clean":"2024-04-25T00:43:00.014387+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:43:00.014387+0000","last_undegraded":"2024-04-25T00:43:00.014387+0000","last_fullsized":"2024-04-25T00:43:00.014387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013174+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:43:00.013174+0000","last_peered":"2024-04-25T00:43:00.013174+0000","last_clean":"2024-04-25T00:43:00.013174+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:43:00.013174+0000","last_undegraded":"2024-04-25T00:43:00.013174+0000","last_fullsized":"2024-04-25T00:43:00.013174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659268+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:42:59.659268+0000","last_peered":"2024-04-25T00:42:59.659268+0000","last_clean":"2024-04-25T00:42:59.659268+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:42:59.659268+0000","last_undegraded":"2024-04-25T00:42:59.659268+0000","last_fullsized":"2024-04-25T00:42:59.659268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656849+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:42:59.656849+0000","last_peered":"2024-04-25T00:42:59.656849+0000","last_clean":"2024-04-25T00:42:59.656849+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:42:59.656849+0000","last_undegraded":"2024-04-25T00:42:59.656849+0000","last_fullsized":"2024-04-25T00:42:59.656849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941104+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:42:59.941104+0000","last_peered":"2024-04-25T00:42:59.941104+0000","last_clean":"2024-04-25T00:42:59.941104+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:42:59.941104+0000","last_undegraded":"2024-04-25T00:42:59.941104+0000","last_fullsized":"2024-04-25T00:42:59.941104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657161+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:42:59.657161+0000","last_peered":"2024-04-25T00:42:59.657161+0000","last_clean":"2024-04-25T00:42:59.657161+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:42:59.657161+0000","last_undegraded":"2024-04-25T00:42:59.657161+0000","last_fullsized":"2024-04-25T00:42:59.657161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941098+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:42:59.941098+0000","last_peered":"2024-04-25T00:42:59.941098+0000","last_clean":"2024-04-25T00:42:59.941098+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:42:59.941098+0000","last_undegraded":"2024-04-25T00:42:59.941098+0000","last_fullsized":"2024-04-25T00:42:59.941098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015070+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:43:00.015070+0000","last_peered":"2024-04-25T00:43:00.015070+0000","last_clean":"2024-04-25T00:43:00.015070+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:43:00.015070+0000","last_undegraded":"2024-04-25T00:43:00.015070+0000","last_fullsized":"2024-04-25T00:43:00.015070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662834+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:42:59.662834+0000","last_peered":"2024-04-25T00:42:59.662834+0000","last_clean":"2024-04-25T00:42:59.662834+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:42:59.662834+0000","last_undegraded":"2024-04-25T00:42:59.662834+0000","last_fullsized":"2024-04-25T00:42:59.662834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014334+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:43:00.014334+0000","last_peered":"2024-04-25T00:43:00.014334+0000","last_clean":"2024-04-25T00:43:00.014334+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:43:00.014334+0000","last_undegraded":"2024-04-25T00:43:00.014334+0000","last_fullsized":"2024-04-25T00:43:00.014334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014890+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:43:00.014890+0000","last_peered":"2024-04-25T00:43:00.014890+0000","last_clean":"2024-04-25T00:43:00.014890+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:43:00.014890+0000","last_undegraded":"2024-04-25T00:43:00.014890+0000","last_fullsized":"2024-04-25T00:43:00.014890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013948+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:43:00.013948+0000","last_peered":"2024-04-25T00:43:00.013948+0000","last_clean":"2024-04-25T00:43:00.013948+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:43:00.013948+0000","last_undegraded":"2024-04-25T00:43:00.013948+0000","last_fullsized":"2024-04-25T00:43:00.013948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015334+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:43:00.015334+0000","last_peered":"2024-04-25T00:43:00.015334+0000","last_clean":"2024-04-25T00:43:00.015334+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:43:00.015334+0000","last_undegraded":"2024-04-25T00:43:00.015334+0000","last_fullsized":"2024-04-25T00:43:00.015334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014217+0000","last_change":"2024-04-25T00:42:58.700939+0000","last_active":"2024-04-25T00:43:00.014217+0000","last_peered":"2024-04-25T00:43:00.014217+0000","last_clean":"2024-04-25T00:43:00.014217+0000","last_became_active":"2024-04-25T00:42:58.700177+0000","last_became_peered":"2024-04-25T00:42:58.700177+0000","last_unstale":"2024-04-25T00:43:00.014217+0000","last_undegraded":"2024-04-25T00:43:00.014217+0000","last_fullsized":"2024-04-25T00:43:00.014217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:36.086387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941542+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:42:59.941542+0000","last_peered":"2024-04-25T00:42:59.941542+0000","last_clean":"2024-04-25T00:42:59.941542+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:42:59.941542+0000","last_undegraded":"2024-04-25T00:42:59.941542+0000","last_fullsized":"2024-04-25T00:42:59.941542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941612+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:42:59.941612+0000","last_peered":"2024-04-25T00:42:59.941612+0000","last_clean":"2024-04-25T00:42:59.941612+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:42:59.941612+0000","last_undegraded":"2024-04-25T00:42:59.941612+0000","last_fullsized":"2024-04-25T00:42:59.941612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015091+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:43:00.015091+0000","last_peered":"2024-04-25T00:43:00.015091+0000","last_clean":"2024-04-25T00:43:00.015091+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:43:00.015091+0000","last_undegraded":"2024-04-25T00:43:00.015091+0000","last_fullsized":"2024-04-25T00:43:00.015091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:43:03.078 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:43:03.078 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:43:03.078 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:43:03.078 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T00:43:03.078 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T00:43:03.150 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-04-25T00:43:03.149+0000 7f90d899f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:43:03.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bfa467640 1 Processor -- start 2024-04-25T00:43:03.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bfa467640 1 -- start start 2024-04-25T00:43:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bfa467640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0bf40aa0c0 0x7f0bf40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bfa467640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf406d1b0 con 0x7f0bf40aa550 2024-04-25T00:43:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bfa467640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf406d490 con 0x7f0bf415bf50 2024-04-25T00:43:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bfa467640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf415ca40 con 0x7f0bf40aa0c0 2024-04-25T00:43:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf37fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0bf40aa0c0 0x7f0bf40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf8a04640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0bf40aa550 0x7f0bf406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46608/0 (socket says 172.21.15.129:46608) 2024-04-25T00:43:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf8a04640 1 -- 172.21.15.129:0/3095837207 learned_addr learned my addr 172.21.15.129:0/3095837207 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf37fe640 1 -- 172.21.15.129:0/3095837207 >> v1:172.21.15.196:6789/0 conn(0x7f0bf415bf50 legacy=0x7f0bf415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf37fe640 1 -- 172.21.15.129:0/3095837207 >> v1:172.21.15.129:6789/0 conn(0x7f0bf40aa550 legacy=0x7f0bf406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf37fe640 1 -- 172.21.15.129:0/3095837207 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf406d770 con 0x7f0bf40aa0c0 2024-04-25T00:43:03.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.224+0000 7f0bf2ffd640 1 -- 172.21.15.129:0/3095837207 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641019144 0 0) 0x7f0bf406d1b0 con 0x7f0bf40aa550 2024-04-25T00:43:03.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf37fe640 1 --2- 172.21.15.129:0/3095837207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf40aa0c0 0x7f0bf40ccb50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0be80097c0 tx=0x7f0be801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4c85bd3691c1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf2ffd640 1 -- 172.21.15.129:0/3095837207 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be80ab650 con 0x7f0bf40aa0c0 2024-04-25T00:43:03.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf2ffd640 1 -- 172.21.15.129:0/3095837207 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be80b0070 con 0x7f0bf40aa0c0 2024-04-25T00:43:03.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf2ffd640 1 -- 172.21.15.129:0/3095837207 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be80aba10 con 0x7f0bf40aa0c0 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- 172.21.15.129:0/3095837207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf40aa0c0 msgr2=0x7f0bf40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 --2- 172.21.15.129:0/3095837207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf40aa0c0 0x7f0bf40ccb50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0be80097c0 tx=0x7f0be801c430 comp rx=0 tx=0).stop 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- 172.21.15.129:0/3095837207 shutdown_connections 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 --2- 172.21.15.129:0/3095837207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf40aa0c0 0x7f0bf40ccb50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- 172.21.15.129:0/3095837207 >> 172.21.15.129:0/3095837207 conn(0x7f0bf40b14a0 msgr2=0x7f0bf40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- 172.21.15.129:0/3095837207 shutdown_connections 2024-04-25T00:43:03.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- 172.21.15.129:0/3095837207 wait complete. 2024-04-25T00:43:03.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 Processor -- start 2024-04-25T00:43:03.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- start start 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bf40aa0c0 0x7f0bf4072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bf40aa550 0x7f0bf41fa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 0x7f0bf41464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf40cd4a0 con 0x7f0bf415bf50 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf406cf70 con 0x7f0bf40aa550 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bfa467640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf416a040 con 0x7f0bf40aa0c0 2024-04-25T00:43:03.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 0x7f0bf41464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 0x7f0bf41464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42654/0 (socket says 172.21.15.129:42654) 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 -- 172.21.15.129:0/4277338201 learned_addr learned my addr 172.21.15.129:0/4277338201 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf3fff640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bf40aa0c0 0x7f0bf4072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf37fe640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bf40aa550 0x7f0bf41fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 -- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bf40aa0c0 msgr2=0x7f0bf4072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bf40aa0c0 0x7f0bf4072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 -- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bf40aa550 msgr2=0x7f0bf41fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bf40aa550 0x7f0bf41fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 -- 172.21.15.129:0/4277338201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf4070730 con 0x7f0bf415bf50 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf37fe640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bf40aa550 0x7f0bf41fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.228+0000 7f0bf8a04640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 0x7f0bf41464a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00c4b0 tx=0x7f0bec00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec018070 con 0x7f0bf415bf50 2024-04-25T00:43:03.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec0074e0 con 0x7f0bf415bf50 2024-04-25T00:43:03.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf414f940 con 0x7f0bf415bf50 2024-04-25T00:43:03.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec004a30 con 0x7f0bf415bf50 2024-04-25T00:43:03.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf414ff00 con 0x7f0bf415bf50 2024-04-25T00:43:03.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf4070900 con 0x7f0bf415bf50 2024-04-25T00:43:03.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0bec01b070 con 0x7f0bf415bf50 2024-04-25T00:43:03.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf0ff9640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0be00678c0 0x7f0be0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf3fff640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0be00678c0 0x7f0be0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bec084f10 con 0x7f0bf415bf50 2024-04-25T00:43:03.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.232+0000 7f0bf3fff640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0be00678c0 0x7f0be0069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009f20 tx=0x7f0be40094d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.236+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0bec055a50 con 0x7f0bf415bf50 2024-04-25T00:43:03.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.420+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0bf4150240 con 0x7f0bf415bf50 2024-04-25T00:43:03.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bf0ff9640 1 -- 172.21.15.129:0/4277338201 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166893 (secure 0 0 0) 0x7f0bec055350 con 0x7f0bf415bf50 2024-04-25T00:43:03.433 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:03.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0be00678c0 msgr2=0x7f0be0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0be00678c0 0x7f0be0069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009f20 tx=0x7f0be40094d0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 msgr2=0x7f0bf41464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 0x7f0bf41464a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00c4b0 tx=0x7f0bec00c970 comp rx=0 tx=0).stop 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 shutdown_connections 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0be00678c0 0x7f0be0069d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bf415bf50 0x7f0bf41464a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bf40aa550 0x7f0bf41fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 --2- 172.21.15.129:0/4277338201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bf40aa0c0 0x7f0bf4072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 >> 172.21.15.129:0/4277338201 conn(0x7f0bf40b14a0 msgr2=0x7f0bf40a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 shutdown_connections 2024-04-25T00:43:03.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.428+0000 7f0bfa467640 1 -- 172.21.15.129:0/4277338201 wait complete. 2024-04-25T00:43:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2280039152},{"type":"v1","addr":"172.21.15.129:6833","nonce":2280039152}]},"active_addr":"172.21.15.129:6833/2280039152","active_change":"2024-04-25T00:42:38.594909+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3065892659}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":501912712}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2257653450}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3322091195}]}]} 2024-04-25T00:43:03.453 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T00:43:03.453 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T00:43:03.453 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:43:03.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.568+0000 7f1efb0c8640 1 Processor -- start 2024-04-25T00:43:03.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.568+0000 7f1efb0c8640 1 -- start start 2024-04-25T00:43:03.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1ef40ccdb0 0x7f1ef40723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ef415c7f0 0x7f1ef415c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef4072aa0 con 0x7f1ef40cd190 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef4070690 con 0x7f1ef40ccdb0 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef40708e0 con 0x7f1ef415c7f0 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1ef40ccdb0 0x7f1ef40723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1ef40ccdb0 0x7f1ef40723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42670/0 (socket says 172.21.15.129:42670) 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 -- 172.21.15.129:0/2552829973 learned_addr learned my addr 172.21.15.129:0/2552829973 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef3fff640 1 --2- 172.21.15.129:0/2552829973 >> v2:172.21.15.196:3300/0 conn(0x7f1ef415c7f0 0x7f1ef415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 -- 172.21.15.129:0/2552829973 >> v2:172.21.15.196:3300/0 conn(0x7f1ef415c7f0 msgr2=0x7f1ef415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 --2- 172.21.15.129:0/2552829973 >> v2:172.21.15.196:3300/0 conn(0x7f1ef415c7f0 0x7f1ef415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 -- 172.21.15.129:0/2552829973 >> v1:172.21.15.196:6789/0 conn(0x7f1ef40cd190 legacy=0x7f1ef4158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:03.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 -- 172.21.15.129:0/2552829973 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef4070bc0 con 0x7f1ef40ccdb0 2024-04-25T00:43:03.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef8e64640 1 --2- 172.21.15.129:0/2552829973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef40723e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4072d80 tx=0x7f1ee401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294e6912906f8484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef37fe640 1 -- 172.21.15.129:0/2552829973 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40b0070 con 0x7f1ef40ccdb0 2024-04-25T00:43:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef37fe640 1 -- 172.21.15.129:0/2552829973 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee40ab400 con 0x7f1ef40ccdb0 2024-04-25T00:43:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1ef37fe640 1 -- 172.21.15.129:0/2552829973 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40ab7d0 con 0x7f1ef40ccdb0 2024-04-25T00:43:03.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 -- 172.21.15.129:0/2552829973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40ccdb0 msgr2=0x7f1ef40723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.572+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/2552829973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef40723e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4072d80 tx=0x7f1ee401c450 comp rx=0 tx=0).stop 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/2552829973 shutdown_connections 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/2552829973 >> v2:172.21.15.196:3300/0 conn(0x7f1ef415c7f0 0x7f1ef415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/2552829973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef40723e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/2552829973 >> 172.21.15.129:0/2552829973 conn(0x7f1ef40b14a0 msgr2=0x7f1ef40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/2552829973 shutdown_connections 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/2552829973 wait complete. 2024-04-25T00:43:03.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 Processor -- start 2024-04-25T00:43:03.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- start start 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 0x7f1ef40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef415c7f0 0x7f1ef4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef406ea10 con 0x7f1ef40cd190 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef406e890 con 0x7f1ef40ccdb0 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef406eb90 con 0x7f1ef415c7f0 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 0x7f1ef40ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 0x7f1ef40ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42680/0 (socket says 172.21.15.129:42680) 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 -- 172.21.15.129:0/1506996120 learned_addr learned my addr 172.21.15.129:0/1506996120 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef8e64640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef9665640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef415c7f0 0x7f1ef4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 -- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef415c7f0 msgr2=0x7f1ef4156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef415c7f0 0x7f1ef4156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 -- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef40ccdb0 msgr2=0x7f1ef413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef413fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 -- 172.21.15.129:0/1506996120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef40b00b0 con 0x7f1ef40cd190 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef8e64640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef413fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef9665640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef415c7f0 0x7f1ef4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef3fff640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 0x7f1ef40ad060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d5f0 tx=0x7f1ee800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80125f0 con 0x7f1ef40cd190 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8019070 con 0x7f1ef40cd190 2024-04-25T00:43:03.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef40b0390 con 0x7f1ef40cd190 2024-04-25T00:43:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80129b0 con 0x7f1ef40cd190 2024-04-25T00:43:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef40b0950 con 0x7f1ef40cd190 2024-04-25T00:43:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.576+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef4062e50 con 0x7f1ef40cd190 2024-04-25T00:43:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.580+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ee8012b50 con 0x7f1ef40cd190 2024-04-25T00:43:03.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.580+0000 7f1ef17fa640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ee00678c0 0x7f1ee0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.580+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1ee80860d0 con 0x7f1ef40cd190 2024-04-25T00:43:03.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.580+0000 7f1ef8e64640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ee00678c0 0x7f1ee0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.580+0000 7f1ef8e64640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ee00678c0 0x7f1ee0069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40ad000 tx=0x7f1ee4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.584+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ee8056c10 con 0x7f1ef40cd190 2024-04-25T00:43:03.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.748+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ef4075320 con 0x7f1ef40cd190 2024-04-25T00:43:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1ef17fa640 1 -- 172.21.15.129:0/1506996120 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f1ee8056510 con 0x7f1ef40cd190 2024-04-25T00:43:03.755 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:03.755 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:59.648406+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ee00678c0 msgr2=0x7f1ee0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ee00678c0 0x7f1ee0069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40ad000 tx=0x7f1ee4008040 comp rx=0 tx=0).stop 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 msgr2=0x7f1ef40ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 0x7f1ef40ad060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d5f0 tx=0x7f1ee800dab0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 shutdown_connections 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ee00678c0 0x7f1ee0069d80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ef415c7f0 0x7f1ef4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ef40cd190 0x7f1ef40ad060 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d5f0 tx=0x7f1ee800dab0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 --2- 172.21.15.129:0/1506996120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef40ccdb0 0x7f1ef413fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 >> 172.21.15.129:0/1506996120 conn(0x7f1ef40b14a0 msgr2=0x7f1ef40cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 shutdown_connections 2024-04-25T00:43:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.752+0000 7f1efb0c8640 1 -- 172.21.15.129:0/1506996120 wait complete. 2024-04-25T00:43:03.774 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T00:43:03.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:43:03.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.884+0000 7f8675632640 1 Processor -- start 2024-04-25T00:43:03.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.884+0000 7f8675632640 1 -- start start 2024-04-25T00:43:03.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f8675632640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f867015bf50 0x7f867015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f8675632640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867006d1b0 con 0x7f86700aa0c0 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f8675632640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867006d490 con 0x7f86700cd260 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f8675632640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f867006d770 con 0x7f867015bf50 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866effd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f867015bf50 0x7f867015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866e7fc640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f86700aa0c0 0x7f86700ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46612/0 (socket says 172.21.15.129:46612) 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866e7fc640 1 -- 172.21.15.129:0/4001175105 learned_addr learned my addr 172.21.15.129:0/4001175105 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866effd640 1 -- 172.21.15.129:0/4001175105 >> v1:172.21.15.196:6789/0 conn(0x7f86700cd260 legacy=0x7f867006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866effd640 1 -- 172.21.15.129:0/4001175105 >> v1:172.21.15.129:6789/0 conn(0x7f86700aa0c0 legacy=0x7f86700ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866dffb640 1 -- 172.21.15.129:0/4001175105 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657777941 0 0) 0x7f867006d1b0 con 0x7f86700aa0c0 2024-04-25T00:43:03.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.888+0000 7f866effd640 1 -- 172.21.15.129:0/4001175105 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867006da50 con 0x7f867015bf50 2024-04-25T00:43:03.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f866effd640 1 --2- 172.21.15.129:0/4001175105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f867015c330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86600097c0 tx=0x7f866001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628da7eaf9e17844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f866dffb640 1 -- 172.21.15.129:0/4001175105 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86600b2070 con 0x7f867015bf50 2024-04-25T00:43:03.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f866dffb640 1 -- 172.21.15.129:0/4001175105 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86600ad680 con 0x7f867015bf50 2024-04-25T00:43:03.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f866dffb640 1 -- 172.21.15.129:0/4001175105 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86600ada50 con 0x7f867015bf50 2024-04-25T00:43:03.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 -- 172.21.15.129:0/4001175105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 msgr2=0x7f867015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 --2- 172.21.15.129:0/4001175105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f867015c330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f86600097c0 tx=0x7f866001c470 comp rx=0 tx=0).stop 2024-04-25T00:43:03.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 -- 172.21.15.129:0/4001175105 shutdown_connections 2024-04-25T00:43:03.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 --2- 172.21.15.129:0/4001175105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f867015c330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 -- 172.21.15.129:0/4001175105 >> 172.21.15.129:0/4001175105 conn(0x7f86700b14a0 msgr2=0x7f86700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:03.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 -- 172.21.15.129:0/4001175105 shutdown_connections 2024-04-25T00:43:03.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.892+0000 7f8675632640 1 -- 172.21.15.129:0/4001175105 wait complete. 2024-04-25T00:43:03.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 Processor -- start 2024-04-25T00:43:03.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- start start 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700aa0c0 0x7f867014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86700cd260 0x7f8670140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f86700b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f867016a1c0 con 0x7f867015bf50 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f867016a040 con 0x7f86700aa0c0 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f867016a340 con 0x7f86700cd260 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f86700b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f86700b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42702/0 (socket says 172.21.15.129:42702) 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 -- 172.21.15.129:0/751296339 learned_addr learned my addr 172.21.15.129:0/751296339 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:03.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866e7fc640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86700cd260 0x7f8670140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866effd640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700aa0c0 0x7f867014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 -- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86700cd260 msgr2=0x7f8670140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86700cd260 0x7f8670140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 -- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700aa0c0 msgr2=0x7f867014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700aa0c0 0x7f867014a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 -- 172.21.15.129:0/751296339 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86700707c0 con 0x7f867015bf50 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f866f7fe640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f86700b01a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f866800c7d0 tx=0x7f866800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668003b80 con 0x7f867015bf50 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668003d20 con 0x7f867015bf50 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800aa30 con 0x7f867015bf50 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8670070aa0 con 0x7f867015bf50 2024-04-25T00:43:03.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8670071060 con 0x7f867015bf50 2024-04-25T00:43:03.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8670062e50 con 0x7f867015bf50 2024-04-25T00:43:03.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.896+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f866800abd0 con 0x7f867015bf50 2024-04-25T00:43:03.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.900+0000 7f86577fe640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f865c0678c0 0x7f865c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:03.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.900+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8668085b20 con 0x7f867015bf50 2024-04-25T00:43:03.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.900+0000 7f866effd640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f865c0678c0 0x7f865c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:03.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.900+0000 7f866effd640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f865c0678c0 0x7f865c069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86600af000 tx=0x7f86600a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:03.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:03.904+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f86680566f0 con 0x7f867015bf50 2024-04-25T00:43:04.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.056+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8670075320 con 0x7f867015bf50 2024-04-25T00:43:04.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f86577fe640 1 -- 172.21.15.129:0/751296339 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f8668055ff0 con 0x7f867015bf50 2024-04-25T00:43:04.064 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:04.064 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:59.648406+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:43:04.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f865c0678c0 msgr2=0x7f865c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f865c0678c0 0x7f865c069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86600af000 tx=0x7f86600a7000 comp rx=0 tx=0).stop 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 msgr2=0x7f86700b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f86700b01a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f866800c7d0 tx=0x7f866800cc90 comp rx=0 tx=0).stop 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 shutdown_connections 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f865c0678c0 0x7f865c069d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86700cd260 0x7f8670140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f867015bf50 0x7f86700b01a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 --2- 172.21.15.129:0/751296339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86700aa0c0 0x7f867014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 >> 172.21.15.129:0/751296339 conn(0x7f86700b14a0 msgr2=0x7f867006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 shutdown_connections 2024-04-25T00:43:04.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.060+0000 7f8675632640 1 -- 172.21.15.129:0/751296339 wait complete. 2024-04-25T00:43:04.082 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T00:43:04.082 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T00:43:04.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T00:43:04.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T00:43:04.084 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T00:43:04.084 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T00:43:04.084 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T00:43:04.085 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T00:43:04.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ef27c640 1 Processor -- start 2024-04-25T00:43:04.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ef27c640 1 -- start start 2024-04-25T00:43:04.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ef27c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70e00f04a0 0x7f70e00f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ef27c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e0034220 con 0x7f70e0033e40 2024-04-25T00:43:04.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ef27c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e004fa30 con 0x7f70e0047c20 2024-04-25T00:43:04.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ef27c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f70e0050210 con 0x7f70e00f04a0 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ee27a640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f70e0033e40 0x7f70e00eb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46618/0 (socket says 172.21.15.129:46618) 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70ee27a640 1 -- 172.21.15.129:0/456696884 learned_addr learned my addr 172.21.15.129:0/456696884 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.252+0000 7f70eea7b640 1 --2- 172.21.15.129:0/456696884 >> v2:172.21.15.196:3300/0 conn(0x7f70e00f04a0 0x7f70e00f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162734126 0 0) 0x7f70e0034220 con 0x7f70e0033e40 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70d4003530 con 0x7f70e0033e40 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 39165364 0 0) 0x7f70d4003530 con 0x7f70e0033e40 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 >> v2:172.21.15.196:3300/0 conn(0x7f70e00f04a0 msgr2=0x7f70e00f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 --2- 172.21.15.129:0/456696884 >> v2:172.21.15.196:3300/0 conn(0x7f70e00f04a0 0x7f70e00f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 >> v1:172.21.15.196:6789/0 conn(0x7f70e0047c20 legacy=0x7f70e00eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e004f2e0 con 0x7f70e0033e40 2024-04-25T00:43:04.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f70e8074390 con 0x7f70e0033e40 2024-04-25T00:43:04.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 >> v1:172.21.15.129:6789/0 conn(0x7f70e0033e40 legacy=0x7f70e00eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70d40039c0 0x7f70d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004320 0x7f70d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4004f30 0x7f70d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70e0050210 con 0x7f70d4004f30 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70d400bf10 con 0x7f70d4004320 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70d400c090 con 0x7f70d40039c0 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eea7b640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4004f30 0x7f70d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70ee27a640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70d40039c0 0x7f70d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eda79640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004320 0x7f70d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eea7b640 1 -- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70d40039c0 msgr2=0x7f70d4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eea7b640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70d40039c0 0x7f70d4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eea7b640 1 -- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004320 msgr2=0x7f70d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eea7b640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004320 0x7f70d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f70eea7b640 1 -- 172.21.15.129:0/456696884 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d4003530 con 0x7f70d4004f30 2024-04-25T00:43:04.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f854a69e640 1 Processor -- start 2024-04-25T00:43:04.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.256+0000 7f854a69e640 1 -- start start 2024-04-25T00:43:04.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70eea7b640 1 -- 172.21.15.129:0/456696884 reap_dead start 2024-04-25T00:43:04.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 Processor -- start 2024-04-25T00:43:04.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70eea7b640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4004f30 0x7f70d400b9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ae0 tx=0x7f70e401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe384c344b62607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854a69e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f853c0501b0 0x7f853c006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854a69e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f853c033d00 0x7f853c006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854a69e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f853c054120 con 0x7f853c0df2b0 2024-04-25T00:43:04.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854a69e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f853c0df690 con 0x7f853c033d00 2024-04-25T00:43:04.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854a69e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f853c054400 con 0x7f853c0501b0 2024-04-25T00:43:04.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e40096b0 con 0x7f70d4004f30 2024-04-25T00:43:04.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e4004020 con 0x7f70d4004f30 2024-04-25T00:43:04.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ed278640 1 -- 172.21.15.129:0/456696884 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004410 con 0x7f70d4004f30 2024-04-25T00:43:04.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 Processor -- start 2024-04-25T00:43:04.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 -- start start 2024-04-25T00:43:04.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8548e9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f853c0501b0 0x7f853c006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854969c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f853c0df2b0 0x7f853c04faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46630/0 (socket says 172.21.15.129:46630) 2024-04-25T00:43:04.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f854969c640 1 -- 172.21.15.129:0/337219777 learned_addr learned my addr 172.21.15.129:0/337219777 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 -- 172.21.15.129:0/456696884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4004f30 msgr2=0x7f70d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4004f30 0x7f70d400b9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ae0 tx=0x7f70e401cab0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 -- start start 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f415ff90 0x7f34f4163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f34f4160370 0x7f34f40b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 0x7f34f41661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f34f40b3060 con 0x7f34f40b2c10 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34f40cd390 con 0x7f34f4160370 2024-04-25T00:43:04.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34fa394640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34f40cc780 con 0x7f34f415ff90 2024-04-25T00:43:04.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8549e9d640 1 --2- 172.21.15.129:0/337219777 >> v2:172.21.15.129:3300/0 conn(0x7f853c033d00 0x7f853c006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8548e9b640 1 -- 172.21.15.129:0/337219777 >> v2:172.21.15.129:3300/0 conn(0x7f853c033d00 msgr2=0x7f853c006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8548e9b640 1 --2- 172.21.15.129:0/337219777 >> v2:172.21.15.129:3300/0 conn(0x7f853c033d00 0x7f853c006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8548e9b640 1 -- 172.21.15.129:0/337219777 >> v1:172.21.15.129:6789/0 conn(0x7f853c0df2b0 legacy=0x7f853c04faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8548e9b640 1 -- 172.21.15.129:0/337219777 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c0546e0 con 0x7f853c0501b0 2024-04-25T00:43:04.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f8549e9d640 1 --2- 172.21.15.129:0/337219777 >> v2:172.21.15.129:3300/0 conn(0x7f853c033d00 0x7f853c006a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:04.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7ff1063ec640 1 Processor -- start 2024-04-25T00:43:04.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f9b93640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 0x7f34f41661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f9b93640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 0x7f34f41661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42716/0 (socket says 172.21.15.129:42716) 2024-04-25T00:43:04.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f9b93640 1 -- 172.21.15.129:0/3553031763 learned_addr learned my addr 172.21.15.129:0/3553031763 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3553031763 >> v2:172.21.15.196:3300/0 conn(0x7f34f4160370 0x7f34f40b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5148006900 0x7f5148006d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 -- 172.21.15.129:0/456696884 shutdown_connections 2024-04-25T00:43:04.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 --2- 172.21.15.129:0/456696884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70d4004f30 0x7f70d400b9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 -- 172.21.15.129:0/456696884 >> 172.21.15.129:0/456696884 conn(0x7f70e0009fc0 msgr2=0x7f70e000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5148053330 0x7f51480537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f9392640 1 --2- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f415ff90 0x7f34f4163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5148007240 con 0x7f5148034160 2024-04-25T00:43:04.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f51480e0aa0 con 0x7f5148006900 2024-04-25T00:43:04.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51565f0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5148052ea0 con 0x7f5148053330 2024-04-25T00:43:04.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f2ff9ef8640 1 Processor -- start 2024-04-25T00:43:04.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f8b91640 1 -- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f415ff90 msgr2=0x7f34f4163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7ff1063ec640 1 -- start start 2024-04-25T00:43:04.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51555ee640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5148034160 0x7f51480e3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:54558/0 (socket says 172.21.15.129:54558) 2024-04-25T00:43:04.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f51555ee640 1 -- 172.21.15.129:0/3856054299 learned_addr learned my addr 172.21.15.129:0/3856054299 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f5154ded640 1 --2- 172.21.15.129:0/3856054299 >> v2:172.21.15.129:3300/0 conn(0x7f5148006900 0x7f5148006d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f2ff9ef8640 1 -- start start 2024-04-25T00:43:04.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f415ff90 0x7f34f4163640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f5157852640 1 --2- 172.21.15.129:0/3856054299 >> v2:172.21.15.196:3300/0 conn(0x7f5148053330 0x7f51480537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f8b91640 1 -- 172.21.15.129:0/3553031763 >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 msgr2=0x7f34f41661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 -- 172.21.15.129:0/456696884 shutdown_connections 2024-04-25T00:43:04.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f70ef27c640 1 -- 172.21.15.129:0/456696884 wait complete. 2024-04-25T00:43:04.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3553031763 >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 0x7f34f41661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f34f8b91640 1 -- 172.21.15.129:0/3553031763 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f40cca60 con 0x7f34f4160370 2024-04-25T00:43:04.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f34f9b93640 1 --2- 172.21.15.129:0/3553031763 >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 0x7f34f41661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:43:04.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f5154ded640 1 -- 172.21.15.129:0/3856054299 >> v2:172.21.15.196:3300/0 conn(0x7f5148053330 msgr2=0x7f51480537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f5154ded640 1 --2- 172.21.15.129:0/3856054299 >> v2:172.21.15.196:3300/0 conn(0x7f5148053330 0x7f51480537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f5154ded640 1 -- 172.21.15.129:0/3856054299 >> v1:172.21.15.196:6789/0 conn(0x7f5148034160 legacy=0x7f51480e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.260+0000 7f5154ded640 1 -- 172.21.15.129:0/3856054299 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5148052870 con 0x7f5148006900 2024-04-25T00:43:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f5157852640 1 --2- 172.21.15.129:0/3856054299 >> v2:172.21.15.196:3300/0 conn(0x7f5148053330 0x7f51480537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 Processor -- start 2024-04-25T00:43:04.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff1063ec640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff10015a210 0x7ff1000b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff1063ec640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10006db80 con 0x7ff1000cd1d0 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff1063ec640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1000b12a0 con 0x7ff10006df80 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff1063ec640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff10015e250 con 0x7ff10015a210 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff104989640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff1000cd1d0 0x7ff10006d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46640/0 (socket says 172.21.15.129:46640) 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff104989640 1 -- 172.21.15.129:0/604275109 learned_addr learned my addr 172.21.15.129:0/604275109 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff9ef8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2ff4146fc0 0x7f2ff40cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff9ef8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff406d5e0 0x7f2ff4162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff9ef8640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff4148020 con 0x7f2ff41629d0 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff9ef8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff406da40 con 0x7f2ff4146fc0 2024-04-25T00:43:04.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff9ef8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff40cd050 con 0x7f2ff406d5e0 2024-04-25T00:43:04.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff0fffff640 1 --2- 172.21.15.129:0/604275109 >> v2:172.21.15.196:3300/0 conn(0x7ff10015a210 0x7ff1000b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff3fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f2ff41629d0 0x7f2ff4070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46654/0 (socket says 172.21.15.129:46654) 2024-04-25T00:43:04.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff3fff640 1 -- 172.21.15.129:0/532372375 learned_addr learned my addr 172.21.15.129:0/532372375 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff0fffff640 1 -- 172.21.15.129:0/604275109 >> v1:172.21.15.196:6789/0 conn(0x7ff10006df80 legacy=0x7ff10006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff0fffff640 1 -- 172.21.15.129:0/604275109 >> v1:172.21.15.129:6789/0 conn(0x7ff1000cd1d0 legacy=0x7ff10006d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7ff0fffff640 1 -- 172.21.15.129:0/604275109 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10015e530 con 0x7ff10015a210 2024-04-25T00:43:04.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 Processor -- start 2024-04-25T00:43:04.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 -- start start 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 -- start start 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb40c1625b0 0x7fb40c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb40c0cd990 0x7fb40c070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40c166ef0 con 0x7fb40c071470 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fb40c070aa0 con 0x7fb40c0cd990 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb411156640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb40c070730 con 0x7fb40c1625b0 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f8548e9b640 1 --2- 172.21.15.129:0/337219777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c0501b0 0x7f853c006540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f853c055ac0 tx=0x7f853401c250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=902b561bd9999785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb40c0cd990 0x7fb40c070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb40c0cd990 0x7fb40c070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42724/0 (socket says 172.21.15.129:42724) 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 -- 172.21.15.129:0/2907808717 learned_addr learned my addr 172.21.15.129:0/2907808717 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff2ffd640 1 --2- 172.21.15.129:0/532372375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff406d5e0 0x7f2ff4162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 -- 172.21.15.129:0/2907808717 >> v2:172.21.15.196:3300/0 conn(0x7fb40c1625b0 msgr2=0x7fb40c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 --2- 172.21.15.129:0/2907808717 >> v2:172.21.15.196:3300/0 conn(0x7fb40c1625b0 0x7fb40c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 -- 172.21.15.129:0/2907808717 >> v1:172.21.15.129:6789/0 conn(0x7fb40c071470 legacy=0x7fb40c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7fb40a59c640 1 -- 172.21.15.129:0/2907808717 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1671d0 con 0x7fb40c0cd990 2024-04-25T00:43:04.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f853bfff640 1 -- 172.21.15.129:0/337219777 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340b0070 con 0x7f853c0501b0 2024-04-25T00:43:04.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f853bfff640 1 -- 172.21.15.129:0/337219777 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8534008be0 con 0x7f853c0501b0 2024-04-25T00:43:04.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f853bfff640 1 -- 172.21.15.129:0/337219777 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400a0d0 con 0x7f853c0501b0 2024-04-25T00:43:04.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff27fc640 1 -- 172.21.15.129:0/532372375 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897878788 0 0) 0x7f2fe4001120 con 0x7f2ff41629d0 2024-04-25T00:43:04.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff27fc640 1 -- 172.21.15.129:0/532372375 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fdc003530 con 0x7f2ff41629d0 2024-04-25T00:43:04.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff2ffd640 1 -- 172.21.15.129:0/532372375 >> v2:172.21.15.129:3300/0 conn(0x7f2ff4146fc0 msgr2=0x7f2ff40cc9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:43:04.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff2ffd640 1 --2- 172.21.15.129:0/532372375 >> v2:172.21.15.129:3300/0 conn(0x7f2ff4146fc0 0x7f2ff40cc9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff2ffd640 1 -- 172.21.15.129:0/532372375 >> v1:172.21.15.129:6789/0 conn(0x7f2ff41629d0 legacy=0x7f2ff4070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f2ff2ffd640 1 -- 172.21.15.129:0/532372375 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff40cd330 con 0x7f2ff406d5e0 2024-04-25T00:43:04.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 0x7f34f40b26d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f34f40b39d0 tx=0x7f34f001c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3cdd2004c0af6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f5154ded640 1 --2- 172.21.15.129:0/3856054299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148006d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f514800d850 tx=0x7f5140020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d308e4eb8dba1788 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004f30 0x7f70e00e2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e00de9b0 0x7f70e0037450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e0180c70 0x7f70e0181100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70e004f090 con 0x7f70e0180c70 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70e004fd80 con 0x7f70d4004f30 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ef27c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70e00482c0 con 0x7f70e00de9b0 2024-04-25T00:43:04.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70eea7b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e0180c70 0x7f70e0181100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70eda79640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e00de9b0 0x7f70e0037450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70eea7b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e0180c70 0x7f70e0181100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42738/0 (socket says 172.21.15.129:42738) 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70eea7b640 1 -- 172.21.15.129:0/815699805 learned_addr learned my addr 172.21.15.129:0/815699805 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70eda79640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e00de9b0 0x7f70e0037450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53568/0 (socket says 172.21.15.129:53568) 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f70ee27a640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004f30 0x7f70e00e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 -- 172.21.15.129:0/337219777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c0501b0 msgr2=0x7f853c006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 --2- 172.21.15.129:0/337219777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c0501b0 0x7f853c006540 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f853c055ac0 tx=0x7f853401c250 comp rx=0 tx=0).stop 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 -- 172.21.15.129:0/337219777 shutdown_connections 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 --2- 172.21.15.129:0/337219777 >> v2:172.21.15.129:3300/0 conn(0x7f853c033d00 0x7f853c006a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 --2- 172.21.15.129:0/337219777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c0501b0 0x7f853c006540 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 -- 172.21.15.129:0/337219777 >> 172.21.15.129:0/337219777 conn(0x7f853c009fc0 msgr2=0x7f853c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f5147fff640 1 -- 172.21.15.129:0/3856054299 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400b6070 con 0x7f5148006900 2024-04-25T00:43:04.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f5147fff640 1 -- 172.21.15.129:0/3856054299 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51400a9d50 con 0x7f5148006900 2024-04-25T00:43:04.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f5147fff640 1 -- 172.21.15.129:0/3856054299 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514000b040 con 0x7f5148006900 2024-04-25T00:43:04.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 -- 172.21.15.129:0/337219777 shutdown_connections 2024-04-25T00:43:04.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.264+0000 7f854a69e640 1 -- 172.21.15.129:0/337219777 wait complete. 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70eea7b640 1 -- 172.21.15.129:0/815699805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e00de9b0 msgr2=0x7f70e0037450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70eea7b640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e00de9b0 0x7f70e0037450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70eea7b640 1 -- 172.21.15.129:0/815699805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004f30 msgr2=0x7f70e00e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70eea7b640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70d4004f30 0x7f70e00e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70eea7b640 1 -- 172.21.15.129:0/815699805 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e0034520 con 0x7f70e0180c70 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70eea7b640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e0180c70 0x7f70e0181100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70e401c8e0 tx=0x7f70e4019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34ebfff640 1 -- 172.21.15.129:0/3553031763 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b0070 con 0x7f34f4160370 2024-04-25T00:43:04.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34ebfff640 1 -- 172.21.15.129:0/3553031763 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00ab600 con 0x7f34f4160370 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34ebfff640 1 -- 172.21.15.129:0/3553031763 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00aba10 con 0x7f34f4160370 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 -- 172.21.15.129:0/3856054299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 msgr2=0x7f5148006d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 --2- 172.21.15.129:0/3856054299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148006d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f514800d850 tx=0x7f5140020040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 -- 172.21.15.129:0/3856054299 shutdown_connections 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 --2- 172.21.15.129:0/3856054299 >> v2:172.21.15.196:3300/0 conn(0x7f5148053330 0x7f51480537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 --2- 172.21.15.129:0/3856054299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148006d00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 -- 172.21.15.129:0/3856054299 >> 172.21.15.129:0/3856054299 conn(0x7f5148054ca0 msgr2=0x7f51480550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 -- 172.21.15.129:0/3856054299 shutdown_connections 2024-04-25T00:43:04.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 -- 172.21.15.129:0/3856054299 wait complete. 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 -- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 msgr2=0x7f34f40b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 --2- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 0x7f34f40b26d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f34f40b39d0 tx=0x7f34f001c1e0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e40bb070 con 0x7f70e0180c70 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e40a5ab0 con 0x7f70e0180c70 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4003620 con 0x7f70e0180c70 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e00366b0 con 0x7f70e0180c70 2024-04-25T00:43:04.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e0036c70 con 0x7f70e0180c70 2024-04-25T00:43:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 -- 172.21.15.129:0/3553031763 shutdown_connections 2024-04-25T00:43:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 --2- 172.21.15.129:0/3553031763 >> v2:172.21.15.129:3300/0 conn(0x7f34f40b2c10 0x7f34f41661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 --2- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 0x7f34f40b26d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 --2- 172.21.15.129:0/3553031763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f415ff90 0x7f34f4163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 -- 172.21.15.129:0/3553031763 >> 172.21.15.129:0/3553031763 conn(0x7f34f4067470 msgr2=0x7f34f40d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb40a59c640 1 --2- 172.21.15.129:0/2907808717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c0cd990 0x7fb40c070f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0b2580 tx=0x7fb40001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8301ebdfda466f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 Processor -- start 2024-04-25T00:43:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7ff0fffff640 1 --2- 172.21.15.129:0/604275109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff1000b0d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000a960 tx=0x7ff0f001d6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfecc8b6652ade6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70e00e2f00 con 0x7f70e0180c70 2024-04-25T00:43:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb409d9b640 1 -- 172.21.15.129:0/2907808717 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000b0070 con 0x7fb40c0cd990 2024-04-25T00:43:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb409d9b640 1 -- 172.21.15.129:0/2907808717 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4000ab600 con 0x7fb40c0cd990 2024-04-25T00:43:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7ff0feffd640 1 -- 172.21.15.129:0/604275109 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000b040 con 0x7ff10015a210 2024-04-25T00:43:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7ff0feffd640 1 -- 172.21.15.129:0/604275109 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f00a7700 con 0x7ff10015a210 2024-04-25T00:43:04.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7ff1063ec640 1 -- 172.21.15.129:0/604275109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 msgr2=0x7ff1000b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7ff1063ec640 1 --2- 172.21.15.129:0/604275109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff1000b0d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000a960 tx=0x7ff0f001d6c0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 -- start start 2024-04-25T00:43:04.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 -- 172.21.15.129:0/3553031763 shutdown_connections 2024-04-25T00:43:04.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f34fa394640 1 -- 172.21.15.129:0/3553031763 wait complete. 2024-04-25T00:43:04.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 Processor -- start 2024-04-25T00:43:04.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c033d00 0x7f853c18a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f853c0501b0 0x7f853c0daf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f853c0df2b0 0x7f853c18bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 -- 172.21.15.129:0/2907808717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c0cd990 msgr2=0x7fb40c070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 --2- 172.21.15.129:0/2907808717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c0cd990 0x7fb40c070f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0b2580 tx=0x7fb40001c540 comp rx=0 tx=0).stop 2024-04-25T00:43:04.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854969c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c033d00 0x7f853c18a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854969c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c033d00 0x7f853c18a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:45898/0 (socket says 172.21.15.129:45898) 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f8548e9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f853c0501b0 0x7f853c0daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f8548e9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f853c0501b0 0x7f853c0daf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53572/0 (socket says 172.21.15.129:53572) 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f2ff2ffd640 1 --2- 172.21.15.129:0/532372375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff406d5e0 0x7f2ff4162490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80097c0 tx=0x7f2fe801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f706612bab956a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 -- 172.21.15.129:0/2907808717 shutdown_connections 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 --2- 172.21.15.129:0/2907808717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c0cd990 0x7fb40c070f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 --2- 172.21.15.129:0/2907808717 >> v2:172.21.15.196:3300/0 conn(0x7fb40c1625b0 0x7fb40c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 -- 172.21.15.129:0/2907808717 >> 172.21.15.129:0/2907808717 conn(0x7fb40c0b14a0 msgr2=0x7fb40c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f8549e9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f853c0df2b0 0x7f853c18bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f8549e9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f853c0df2b0 0x7f853c18bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42744/0 (socket says 172.21.15.129:42744) 2024-04-25T00:43:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854969c640 1 -- 172.21.15.129:0/918592729 learned_addr learned my addr 172.21.15.129:0/918592729 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 -- 172.21.15.129:0/2907808717 shutdown_connections 2024-04-25T00:43:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f51565f0640 1 -- start start 2024-04-25T00:43:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 -- 172.21.15.129:0/604275109 shutdown_connections 2024-04-25T00:43:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 --2- 172.21.15.129:0/604275109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff1000b0d60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 -- 172.21.15.129:0/604275109 >> 172.21.15.129:0/604275109 conn(0x7ff100067070 msgr2=0x7ff100067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 -- 172.21.15.129:0/604275109 shutdown_connections 2024-04-25T00:43:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 -- 172.21.15.129:0/604275109 wait complete. 2024-04-25T00:43:04.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148034700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5148034160 0x7f514817daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5148053330 0x7f514817e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5148052b30 con 0x7f5148006900 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 Processor -- start 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5148034160 0x7f514817daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5148034160 0x7f514817daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:45912/0 (socket says 172.21.15.129:45912) 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 -- 172.21.15.129:0/4241316081 learned_addr learned my addr 172.21.15.129:0/4241316081 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f853c04f7e0 con 0x7f853c0df2b0 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f853c04f660 con 0x7f853c033d00 2024-04-25T00:43:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f853c0ec050 con 0x7f853c0501b0 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8548e9b640 1 -- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c033d00 msgr2=0x7f853c18a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8548e9b640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f853c033d00 0x7f853c18a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8548e9b640 1 -- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f853c0df2b0 msgr2=0x7f853c18bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8548e9b640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f853c0df2b0 0x7f853c18bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8548e9b640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c18aae0 con 0x7f853c0501b0 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51480e3f30 con 0x7f5148034160 2024-04-25T00:43:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f514804ff50 con 0x7f5148053330 2024-04-25T00:43:04.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5157852640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5148053330 0x7f514817e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51555ee640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148034700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8548e9b640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f853c0501b0 0x7f853c0daf30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85340aa000 tx=0x7f8534018ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff27fc640 1 -- 172.21.15.129:0/532372375 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80b0070 con 0x7f2ff406d5e0 2024-04-25T00:43:04.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff27fc640 1 -- 172.21.15.129:0/532372375 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe80ab600 con 0x7f2ff406d5e0 2024-04-25T00:43:04.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff27fc640 1 -- 172.21.15.129:0/532372375 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80ab9d0 con 0x7f2ff406d5e0 2024-04-25T00:43:04.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.268+0000 7fb411156640 1 -- 172.21.15.129:0/2907808717 wait complete. 2024-04-25T00:43:04.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f70e40037c0 con 0x7f70e0180c70 2024-04-25T00:43:04.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70deffd640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f70cc0679c0 0x7f70cc069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f70e411ff40 con 0x7f70e0180c70 2024-04-25T00:43:04.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70deffd640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7f70cc06fbe0 0x7f70cc072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 --> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70e40aeec0 con 0x7f70cc06fbe0 2024-04-25T00:43:04.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f70e4120330 con 0x7f70e0180c70 2024-04-25T00:43:04.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70ee27a640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f70cc0679c0 0x7f70cc069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb411156640 1 Processor -- start 2024-04-25T00:43:04.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340ba020 con 0x7f853c0501b0 2024-04-25T00:43:04.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 -- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5148053330 msgr2=0x7f514817e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5148053330 0x7f514817e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 -- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 msgr2=0x7f5148034700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148034700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514804d350 con 0x7f5148034160 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51555ee640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5148006900 0x7f5148034700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c18adc0 con 0x7f853c0501b0 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c18b380 con 0x7f853c0501b0 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8534008be0 con 0x7f853c0501b0 2024-04-25T00:43:04.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340b0040 con 0x7f853c0501b0 2024-04-25T00:43:04.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 Processor -- start 2024-04-25T00:43:04.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb411156640 1 -- start start 2024-04-25T00:43:04.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70ee27a640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f70cc0679c0 0x7f70cc069e80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f70e806a620 tx=0x7f70e8069630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 -- start start 2024-04-25T00:43:04.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7ff1063ec640 1 -- start start 2024-04-25T00:43:04.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5154ded640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5148034160 0x7f514817daf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f514001cc50 tx=0x7f514001cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70eda79640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7f70cc06fbe0 0x7f70cc072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/532372375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff406d5e0 msgr2=0x7f2ff4162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 --2- 172.21.15.129:0/532372375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff406d5e0 0x7f2ff4162490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80097c0 tx=0x7f2fe801c3f0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb411156640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb40c0cd990 0x7fb40c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb411156640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb40c1625b0 0x7fb40c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb411156640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c150ac0 0x7fb40c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/532372375 shutdown_connections 2024-04-25T00:43:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 --2- 172.21.15.129:0/532372375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff406d5e0 0x7f2ff4162490 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 --2- 172.21.15.129:0/532372375 >> v2:172.21.15.129:3300/0 conn(0x7f2ff4146fc0 0x7f2ff40cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/532372375 >> 172.21.15.129:0/532372375 conn(0x7f2ff4072d80 msgr2=0x7f2ff4073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8524001040 con 0x7f853c0501b0 2024-04-25T00:43:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400b6040 con 0x7f5148034160 2024-04-25T00:43:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51400bf070 con 0x7f5148034160 2024-04-25T00:43:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f51565f0640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51480351a0 con 0x7f5148034160 2024-04-25T00:43:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b2c10 0x7f34f415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34f415ff90 0x7f34f40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 0x7f34f4156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34f40cd820 con 0x7f34f415ff90 2024-04-25T00:43:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34f40ce0a0 con 0x7f34f4160370 2024-04-25T00:43:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34fa394640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34f4140680 con 0x7f34f40b2c10 2024-04-25T00:43:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb40b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c150ac0 0x7fb40c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb40b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c150ac0 0x7fb40c0cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42764/0 (socket says 172.21.15.129:42764) 2024-04-25T00:43:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb40ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb40c0cd990 0x7fb40c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb40ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb40c0cd990 0x7fb40c151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:45918/0 (socket says 172.21.15.129:45918) 2024-04-25T00:43:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f70eda79640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7f70cc06fbe0 0x7f70cc072000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/532372375 shutdown_connections 2024-04-25T00:43:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34f8b91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34f415ff90 0x7f34f40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34f8b91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34f415ff90 0x7f34f40b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42772/0 (socket says 172.21.15.129:42772) 2024-04-25T00:43:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34f8b91640 1 -- 172.21.15.129:0/3686632661 learned_addr learned my addr 172.21.15.129:0/3686632661 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f34f9b93640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 0x7f34f4156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7fb40b59e640 1 -- 172.21.15.129:0/431802997 learned_addr learned my addr 172.21.15.129:0/431802997 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40a59c640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb40c1625b0 0x7fb40c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f9392640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b2c10 0x7f34f415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1631a0 con 0x7fb40c150ac0 2024-04-25T00:43:04.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0cd4e0 con 0x7fb40c0cd990 2024-04-25T00:43:04.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400ba670 con 0x7f5148034160 2024-04-25T00:43:04.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c163320 con 0x7fb40c1625b0 2024-04-25T00:43:04.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f51565f0640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5148035760 con 0x7f5148034160 2024-04-25T00:43:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f8b91640 1 -- 172.21.15.129:0/3686632661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b2c10 msgr2=0x7f34f415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34f40b2c10 0x7f34f415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f8b91640 1 -- 172.21.15.129:0/3686632661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 msgr2=0x7f34f4156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f4160370 0x7f34f4156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f8b91640 1 -- 172.21.15.129:0/3686632661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f415eea0 con 0x7f34f415ff90 2024-04-25T00:43:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.272+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/532372375 wait complete. 2024-04-25T00:43:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34f8b91640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34f415ff90 0x7f34f40b1710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f34f0019e10 tx=0x7f34f0018f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85340b4650 con 0x7f853c0501b0 2024-04-25T00:43:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f51565f0640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5148005570 con 0x7f5148034160 2024-04-25T00:43:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40b59e640 1 -- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb40c1625b0 msgr2=0x7fb40c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40b59e640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb40c1625b0 0x7fb40c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40b59e640 1 -- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb40c0cd990 msgr2=0x7fb40c151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40b59e640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb40c0cd990 0x7fb40c151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40b59e640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c151920 con 0x7fb40c150ac0 2024-04-25T00:43:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f2ff9ef8640 1 Processor -- start 2024-04-25T00:43:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b0040 con 0x7f34f415ff90 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f2ff9ef8640 1 -- start start 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== osd.2 v2:172.21.15.129:6824/4081708389 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f70e40aeec0 con 0x7f70cc06fbe0 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f415f180 con 0x7f34f415ff90 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00bd070 con 0x7f34f415ff90 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b6690 con 0x7f34f415ff90 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f40b1c50 con 0x7f34f415ff90 2024-04-25T00:43:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f34df7fe640 1 -- 172.21.15.129:0/3686632661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34cc001040 con 0x7f34f415ff90 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7fb40b59e640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c150ac0 0x7fb40c0cec20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c3c0 tx=0x7fb3fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8539ffb640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f85280678f0 0x7f8528069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8534120480 con 0x7f853c0501b0 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8539ffb640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f852806faf0 0x7f8528071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85340c1ec0 con 0x7f852806faf0 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8534120870 con 0x7f853c0501b0 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f854969c640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f85280678f0 0x7f8528069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8549e9d640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f852806faf0 0x7f8528071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10006df80 0x7ff100204150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1000cd1d0 0x7ff10015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff10015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff10015dee0 con 0x7ff1000cd1d0 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff10015eaf0 con 0x7ff10015a210 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1000b08a0 con 0x7ff10006df80 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff10015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff0ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1000cd1d0 0x7ff10015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff10015d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:45932/0 (socket says 172.21.15.129:45932) 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 -- 172.21.15.129:0/4121393028 learned_addr learned my addr 172.21.15.129:0/4121393028 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 -- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10006df80 msgr2=0x7ff100204150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10006df80 0x7ff100204150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 -- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1000cd1d0 msgr2=0x7ff10015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1000cd1d0 0x7ff10015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 -- 172.21.15.129:0/4121393028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff100156070 con 0x7ff10015a210 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff104989640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff10015d130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800faa0 tx=0x7ff0f800ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f51400df890 con 0x7f5148034160 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5145ffb640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51340678c0 0x7f5134069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51401235f0 con 0x7f5148034160 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f51555ee640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51340678c0 0x7f5134069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5145ffb640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7f513406fa60 0x7f5134071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51400b22a0 con 0x7f513406fa60 2024-04-25T00:43:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5140123a80 con 0x7f5148034160 2024-04-25T00:43:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801a070 con 0x7ff10015a210 2024-04-25T00:43:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f5157852640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7f513406fa60 0x7f5134071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f8549e9d640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f852806faf0 0x7f8528071f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f854969c640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f85280678f0 0x7f8528069db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f854402d710 tx=0x7f854408d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f800d610 con 0x7ff10015a210 2024-04-25T00:43:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10014ac80 con 0x7ff10015a210 2024-04-25T00:43:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800d9c0 con 0x7ff10015a210 2024-04-25T00:43:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001565f0 con 0x7ff10015a210 2024-04-25T00:43:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f51555ee640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51340678c0 0x7f5134069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f514804a550 tx=0x7f514c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f2ff9ef8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff406d5e0 0x7f2ff413bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.276+0000 7f2ff9ef8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff4146fc0 0x7f2ff40afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff9ef8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ff41629d0 0x7f2ff415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 Processor -- start 2024-04-25T00:43:04.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff9ef8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4147520 con 0x7f2ff41629d0 2024-04-25T00:43:04.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff10014aa30 con 0x7ff10015a210 2024-04-25T00:43:04.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff4146fc0 0x7f2ff40afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ff41629d0 0x7f2ff415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff4146fc0 0x7f2ff40afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53616/0 (socket says 172.21.15.129:53616) 2024-04-25T00:43:04.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff2ffd640 1 -- 172.21.15.129:0/1276943878 learned_addr learned my addr 172.21.15.129:0/1276943878 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41473a0 con 0x7f2ff406d5e0 2024-04-25T00:43:04.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff416b170 con 0x7f2ff4146fc0 2024-04-25T00:43:04.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0052d0 con 0x7fb40c150ac0 2024-04-25T00:43:04.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34f00b6830 con 0x7f34f415ff90 2024-04-25T00:43:04.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 -- 172.21.15.129:0/1276943878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff4146fc0 msgr2=0x7f2ff40afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff4146fc0 0x7f2ff40afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 -- 172.21.15.129:0/1276943878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff406d5e0 msgr2=0x7f2ff413bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff406d5e0 0x7f2ff413bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff4140ca0 con 0x7f2ff41629d0 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3fc005470 con 0x7fb40c150ac0 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c151c00 con 0x7fb40c150ac0 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc007260 con 0x7fb40c150ac0 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1521c0 con 0x7fb40c150ac0 2024-04-25T00:43:04.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb40c062e50 con 0x7fb40c150ac0 2024-04-25T00:43:04.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f34d8067940 0x7f34d8069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f34f0120dc0 con 0x7f34f415ff90 2024-04-25T00:43:04.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 -- start start 2024-04-25T00:43:04.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa480b6610 0x7faa4815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4806e510 0x7faa480d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa480cf3a0 con 0x7faa480d06a0 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faa480cf150 con 0x7faa480b6610 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa500cc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa480cfcd0 con 0x7faa4806e510 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34f9392640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f34d8067940 0x7f34d8069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7f34d806fb60 0x7f34d8071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 --> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34f00ace50 con 0x7f34d806fb60 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f34f01212a0 con 0x7f34f415ff90 2024-04-25T00:43:04.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== osd.5 v2:172.21.15.196:6824/546704147 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85340c1ec0 con 0x7f852806faf0 2024-04-25T00:43:04.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa480b6610 0x7faa4815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa480b6610 0x7faa4815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:45950/0 (socket says 172.21.15.129:45950) 2024-04-25T00:43:04.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 -- 172.21.15.129:0/864943884 learned_addr learned my addr 172.21.15.129:0/864943884 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4d667640 1 --2- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4806e510 0x7faa480d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f2ff3fff640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ff41629d0 0x7f2ff415b480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4000c10 tx=0x7f2fe400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34f9b93640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7f34d806fb60 0x7f34d8071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34f9b93640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7f34d806fb60 0x7f34d8071f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff0f801d070 con 0x7ff10015a210 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fcff9640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0ec0678c0 0x7ff0ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff0f8088b00 con 0x7ff10015a210 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fcff9640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7ff0ec06fac0 0x7ff0ec071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 --> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0ec072590 con 0x7ff0ec06fac0 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff0f8088f90 con 0x7ff10015a210 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fffff640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0ec0678c0 0x7ff0ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0ff7fe640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7ff0ec06fac0 0x7ff0ec071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0fffff640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0ec0678c0 0x7ff0ec069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00b0000 tx=0x7ff0f0004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== osd.3 v2:172.21.15.129:6810/1553174573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34f00ace50 con 0x7f34d806fb60 2024-04-25T00:43:04.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7f34f9392640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f34d8067940 0x7f34d8069e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34e400aae0 tx=0x7f34e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7ff0ff7fe640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7ff0ec06fac0 0x7ff0ec071ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 -- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4806e510 msgr2=0x7faa480d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 --2- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4806e510 0x7faa480d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 -- 172.21.15.129:0/864943884 >> v1:172.21.15.129:6789/0 conn(0x7faa480d06a0 legacy=0x7faa4815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.280+0000 7faa4de68640 1 -- 172.21.15.129:0/864943884 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa480cf680 con 0x7faa480b6610 2024-04-25T00:43:04.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f5157852640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7f513406fa60 0x7f5134071e80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40074e0 con 0x7f2ff41629d0 2024-04-25T00:43:04.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff40aebb0 con 0x7f2ff41629d0 2024-04-25T00:43:04.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff40af130 con 0x7f2ff41629d0 2024-04-25T00:43:04.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== osd.7 v2:172.21.15.196:6806/221824364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51400b22a0 con 0x7f513406fa60 2024-04-25T00:43:04.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ff4062e50 con 0x7f2ff41629d0 2024-04-25T00:43:04.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe40047f0 con 0x7f2ff41629d0 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb3fc007400 con 0x7fb40c150ac0 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7fb3f37fe640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb3f80679c0 0x7fb3f8069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb3fc085840 con 0x7fb40c150ac0 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7fb3f37fe640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7fb3f806fbe0 0x7fb3f8072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3f80726b0 con 0x7fb3f806fbe0 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.284+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc085c30 con 0x7fb40c150ac0 2024-04-25T00:43:04.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa4de68640 1 --2- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa480b6610 0x7faa4815ed10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa44006fd0 tx=0x7faa4401e8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2070a7d60673aa6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== osd.6 v2:172.21.15.196:6807/4280234201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0ec072590 con 0x7ff0ec06fac0 2024-04-25T00:43:04.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4004c20 con 0x7f2ff41629d0 2024-04-25T00:43:04.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2fe4011070 con 0x7f2ff41629d0 2024-04-25T00:43:04.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7fb40a59c640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7fb3f806fbe0 0x7fb3f8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7fb40ad9d640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb3f80679c0 0x7fb3f8069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa4ce66640 1 -- 172.21.15.129:0/864943884 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440ad070 con 0x7faa480b6610 2024-04-25T00:43:04.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa4ce66640 1 -- 172.21.15.129:0/864943884 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa44004230 con 0x7faa480b6610 2024-04-25T00:43:04.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa4ce66640 1 -- 172.21.15.129:0/864943884 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440a7760 con 0x7faa480b6610 2024-04-25T00:43:04.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7fb40ad9d640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb3f80679c0 0x7fb3f8069e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb40400a400 tx=0x7fb40400d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2fe00678f0 0x7f2fe0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2fe4085930 con 0x7f2ff41629d0 2024-04-25T00:43:04.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f2fe006fb70 0x7f2fe0071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 -- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa480b6610 msgr2=0x7faa4815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 --2- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa480b6610 0x7faa4815ed10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa44006fd0 tx=0x7faa4401e8f0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 -- 172.21.15.129:0/864943884 shutdown_connections 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 --2- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4806e510 0x7faa480d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 --2- 172.21.15.129:0/864943884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa480b6610 0x7faa4815ed10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 -- 172.21.15.129:0/864943884 >> 172.21.15.129:0/864943884 conn(0x7faa480b2580 msgr2=0x7faa480b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 -- 172.21.15.129:0/864943884 shutdown_connections 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2ff37fe640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2fe00678f0 0x7f2fe0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fe40047f0 con 0x7f2fe006fb70 2024-04-25T00:43:04.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2fe408a2a0 con 0x7f2ff41629d0 2024-04-25T00:43:04.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2ff2ffd640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f2fe006fb70 0x7f2fe0071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2ff2ffd640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f2fe006fb70 0x7f2fe0071f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2ff37fe640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2fe00678f0 0x7f2fe0069db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00ab90 tx=0x7f2fec009230 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7fb40a59c640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7fb3f806fbe0 0x7fb3f8072000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 -- 172.21.15.129:0/864943884 wait complete. 2024-04-25T00:43:04.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 Processor -- start 2024-04-25T00:43:04.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== osd.1 v2:172.21.15.129:6801/1082494394 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fe40047f0 con 0x7f2fe006fb70 2024-04-25T00:43:04.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.288+0000 7faa500cc640 1 -- start start 2024-04-25T00:43:04.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.292+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== osd.4 v2:172.21.15.196:6800/3201055999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3f80726b0 con 0x7fb3f806fbe0 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa500cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa4806e510 0x7faa4814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa500cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa480b6610 0x7faa4814a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa500cc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa48157d30 0x7faa48203630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa500cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa4815ea00 con 0x7faa480b6610 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa500cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa4815e880 con 0x7faa4806e510 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa500cc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faa480b5100 con 0x7faa48157d30 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa480b6610 0x7faa4814a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa480b6610 0x7faa4814a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42798/0 (socket says 172.21.15.129:42798) 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 -- 172.21.15.129:0/1849426212 learned_addr learned my addr 172.21.15.129:0/1849426212 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4de68640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa4806e510 0x7faa4814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4e669640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa48157d30 0x7faa48203630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 -- 172.21.15.129:0/1849426212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa48157d30 msgr2=0x7faa48203630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa48157d30 0x7faa48203630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 -- 172.21.15.129:0/1849426212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa4806e510 msgr2=0x7faa4814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa4806e510 0x7faa4814d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 -- 172.21.15.129:0/1849426212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa48156580 con 0x7faa480b6610 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.296+0000 7faa4d667640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa480b6610 0x7faa4814a5c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa3800e930 tx=0x7faa3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 --> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ff4075320 con 0x7f2fe006fb70 2024-04-25T00:43:04.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38017070 con 0x7faa480b6610 2024-04-25T00:43:04.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3800ca00 con 0x7faa480b6610 2024-04-25T00:43:04.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa48156af0 con 0x7faa480b6610 2024-04-25T00:43:04.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3800cdb0 con 0x7faa480b6610 2024-04-25T00:43:04.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7f2fdbfff640 1 -- 172.21.15.129:0/1276943878 <== osd.1 v2:172.21.15.129:6801/1082494394 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2ff4075320 con 0x7f2fe006fb70 2024-04-25T00:43:04.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.304+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa48156150 con 0x7faa480b6610 2024-04-25T00:43:04.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 --> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f70e00053a0 con 0x7f70cc06fbe0 2024-04-25T00:43:04.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa48062e50 con 0x7faa480b6610 2024-04-25T00:43:04.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70deffd640 1 -- 172.21.15.129:0/815699805 <== osd.2 v2:172.21.15.129:6824/4081708389 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f70e00053a0 con 0x7f70cc06fbe0 2024-04-25T00:43:04.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7f70cc06fbe0 msgr2=0x7f70cc072000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7f70cc06fbe0 0x7f70cc072000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f70cc0679c0 msgr2=0x7f70cc069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f70cc0679c0 0x7f70cc069e80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f70e806a620 tx=0x7f70e8069630 comp rx=0 tx=0).stop 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e0180c70 msgr2=0x7f70e0181100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 --2- 172.21.15.129:0/815699805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e0180c70 0x7f70e0181100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f70e401c8e0 tx=0x7f70e4019850 comp rx=0 tx=0).stop 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70eea7b640 1 -- 172.21.15.129:0/815699805 reap_dead start 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 shutdown_connections 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 >> 172.21.15.129:0/815699805 conn(0x7f70e0009fc0 msgr2=0x7f70e0053ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 shutdown_connections 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f70ef27c640 1 -- 172.21.15.129:0/815699805 wait complete. 2024-04-25T00:43:04.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa3801a070 con 0x7faa480b6610 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f2fe006fb70 msgr2=0x7f2fe0071f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f2fe006fb70 0x7f2fe0071f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2fe00678f0 msgr2=0x7f2fe0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2fe00678f0 0x7f2fe0069db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00ab90 tx=0x7f2fec009230 comp rx=0 tx=0).stop 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ff41629d0 msgr2=0x7f2ff415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 --2- 172.21.15.129:0/1276943878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ff41629d0 0x7f2ff415b480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4000c10 tx=0x7f2fe400c870 comp rx=0 tx=0).stop 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff3fff640 1 -- 172.21.15.129:0/1276943878 reap_dead start 2024-04-25T00:43:04.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 shutdown_connections 2024-04-25T00:43:04.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 >> 172.21.15.129:0/1276943878 conn(0x7f2ff4072d80 msgr2=0x7f2ff4147730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa3e7fc640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa2c0679c0 0x7faa2c069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 shutdown_connections 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa38085f80 con 0x7faa480b6610 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa4de68640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa2c0679c0 0x7faa2c069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa3e7fc640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7faa2c06fbe0 0x7faa2c072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 --> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa38014e50 con 0x7faa2c06fbe0 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa38086370 con 0x7faa480b6610 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa4e669640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7faa2c06fbe0 0x7faa2c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa4de68640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa2c0679c0 0x7faa2c069e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa440b1000 tx=0x7faa440a9070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7faa4e669640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7faa2c06fbe0 0x7faa2c072000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f2ff9ef8640 1 -- 172.21.15.129:0/1276943878 wait complete. 2024-04-25T00:43:04.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f34df7fe640 1 -- 172.21.15.129:0/3686632661 --> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34cc002de0 con 0x7f34d806fb60 2024-04-25T00:43:04.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.308+0000 7f34e9ffb640 1 -- 172.21.15.129:0/3686632661 <== osd.3 v2:172.21.15.129:6810/1553174573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f34cc002de0 con 0x7f34d806fb60 2024-04-25T00:43:04.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 --> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8524002de0 con 0x7f852806faf0 2024-04-25T00:43:04.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== osd.0 v2:172.21.15.129:6800/1911044788 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa38014e50 con 0x7faa2c06fbe0 2024-04-25T00:43:04.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f8539ffb640 1 -- 172.21.15.129:0/918592729 <== osd.5 v2:172.21.15.196:6824/546704147 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8524002de0 con 0x7f852806faf0 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f852806faf0 msgr2=0x7f8528071f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f852806faf0 0x7f8528071f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f85280678f0 msgr2=0x7f8528069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f85280678f0 0x7f8528069db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f854402d710 tx=0x7f854408d040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f853c0501b0 msgr2=0x7f853c0daf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 --2- 172.21.15.129:0/918592729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f853c0501b0 0x7f853c0daf30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85340aa000 tx=0x7f8534018ff0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f8549e9d640 1 -- 172.21.15.129:0/918592729 reap_dead start 2024-04-25T00:43:04.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 shutdown_connections 2024-04-25T00:43:04.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 >> 172.21.15.129:0/918592729 conn(0x7f853c009fc0 msgr2=0x7f853c0365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 shutdown_connections 2024-04-25T00:43:04.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7f854a69e640 1 -- 172.21.15.129:0/918592729 wait complete. 2024-04-25T00:43:04.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.312+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 --> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff100062e50 con 0x7ff0ec06fac0 2024-04-25T00:43:04.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f51565f0640 1 -- 172.21.15.129:0/4241316081 --> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5148036bd0 con 0x7f513406fa60 2024-04-25T00:43:04.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7f34d806fb60 msgr2=0x7f34d8071f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7f34d806fb60 0x7f34d8071f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f34d8067940 msgr2=0x7f34d8069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f34d8067940 0x7f34d8069e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34e400aae0 tx=0x7f34e4008040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34f415ff90 msgr2=0x7f34f40b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 --2- 172.21.15.129:0/3686632661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34f415ff90 0x7f34f40b1710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f34f0019e10 tx=0x7f34f0018f80 comp rx=0 tx=0).stop 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34f9b93640 1 -- 172.21.15.129:0/3686632661 reap_dead start 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 shutdown_connections 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.316+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 >> 172.21.15.129:0/3686632661 conn(0x7f34f4067470 msgr2=0x7f34f406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.320+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 shutdown_connections 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.320+0000 7f34fa394640 1 -- 172.21.15.129:0/3686632661 wait complete. 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.320+0000 7ff0fcff9640 1 -- 172.21.15.129:0/4121393028 <== osd.6 v2:172.21.15.196:6807/4280234201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff100062e50 con 0x7ff0ec06fac0 2024-04-25T00:43:04.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.320+0000 7f5145ffb640 1 -- 172.21.15.129:0/4241316081 <== osd.7 v2:172.21.15.196:6806/221824364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5148036bd0 con 0x7f513406fa60 2024-04-25T00:43:04.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 --> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb40c0b0f00 con 0x7fb3f806fbe0 2024-04-25T00:43:04.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7f513406fa60 msgr2=0x7f5134071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7f513406fa60 0x7f5134071e80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51340678c0 msgr2=0x7f5134069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb3f37fe640 1 -- 172.21.15.129:0/431802997 <== osd.4 v2:172.21.15.196:6800/3201055999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb40c0b0f00 con 0x7fb3f806fbe0 2024-04-25T00:43:04.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51340678c0 0x7f5134069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f514804a550 tx=0x7f514c009040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5148034160 msgr2=0x7f514817daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 --2- 172.21.15.129:0/4241316081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5148034160 0x7f514817daf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f514001cc50 tx=0x7f514001cde0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f5157852640 1 -- 172.21.15.129:0/4241316081 reap_dead start 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 shutdown_connections 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 >> 172.21.15.129:0/4241316081 conn(0x7f5148054ca0 msgr2=0x7f51480ddd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 shutdown_connections 2024-04-25T00:43:04.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7f51337fe640 1 -- 172.21.15.129:0/4241316081 wait complete. 2024-04-25T00:43:04.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7fb3f806fbe0 msgr2=0x7fb3f8072000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7fb3f806fbe0 0x7fb3f8072000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb3f80679c0 msgr2=0x7fb3f8069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb3f80679c0 0x7fb3f8069e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb40400a400 tx=0x7fb40400d040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c150ac0 msgr2=0x7fb40c0cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 --2- 172.21.15.129:0/431802997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40c150ac0 0x7fb40c0cec20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c3c0 tx=0x7fb3fc00c880 comp rx=0 tx=0).stop 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb40b59e640 1 -- 172.21.15.129:0/431802997 reap_dead start 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 shutdown_connections 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 >> 172.21.15.129:0/431802997 conn(0x7fb40c0b14a0 msgr2=0x7fb40c157ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 shutdown_connections 2024-04-25T00:43:04.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.324+0000 7fb411156640 1 -- 172.21.15.129:0/431802997 wait complete. 2024-04-25T00:43:04.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7ff0ec06fac0 msgr2=0x7ff0ec071ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7ff0ec06fac0 0x7ff0ec071ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0ec0678c0 msgr2=0x7ff0ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0ec0678c0 0x7ff0ec069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00b0000 tx=0x7ff0f0004740 comp rx=0 tx=0).stop 2024-04-25T00:43:04.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 msgr2=0x7ff10015d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 --2- 172.21.15.129:0/4121393028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff10015a210 0x7ff10015d130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800faa0 tx=0x7ff0f800ff60 comp rx=0 tx=0).stop 2024-04-25T00:43:04.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff104989640 1 -- 172.21.15.129:0/4121393028 reap_dead start 2024-04-25T00:43:04.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 shutdown_connections 2024-04-25T00:43:04.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 >> 172.21.15.129:0/4121393028 conn(0x7ff100067070 msgr2=0x7ff1000cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 shutdown_connections 2024-04-25T00:43:04.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.332+0000 7ff1063ec640 1 -- 172.21.15.129:0/4121393028 wait complete. 2024-04-25T00:43:04.363 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2024-04-25T00:43:04.363 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T00:43:04.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 --> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa48156ff0 con 0x7faa2c06fbe0 2024-04-25T00:43:04.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa3e7fc640 1 -- 172.21.15.129:0/1849426212 <== osd.0 v2:172.21.15.129:6800/1911044788 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa40009bf0 con 0x7faa2c06fbe0 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7faa2c06fbe0 msgr2=0x7faa2c072000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7faa2c06fbe0 0x7faa2c072000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa2c0679c0 msgr2=0x7faa2c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faa2c0679c0 0x7faa2c069e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa440b1000 tx=0x7faa440a9070 comp rx=0 tx=0).stop 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa480b6610 msgr2=0x7faa4814a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 --2- 172.21.15.129:0/1849426212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa480b6610 0x7faa4814a5c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa3800e930 tx=0x7faa3800edf0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa4e669640 1 -- 172.21.15.129:0/1849426212 reap_dead start 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 shutdown_connections 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 >> 172.21.15.129:0/1849426212 conn(0x7faa480b2580 msgr2=0x7faa4814e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 shutdown_connections 2024-04-25T00:43:04.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.340+0000 7faa500cc640 1 -- 172.21.15.129:0/1849426212 wait complete. 2024-04-25T00:43:04.366 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.366 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T00:43:04.367 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.367 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T00:43:04.368 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2024-04-25T00:43:04.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T00:43:04.369 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.370 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T00:43:04.370 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.371 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T00:43:04.372 INFO:teuthology.orchestra.run.smithi129.stdout:55834574856 2024-04-25T00:43:04.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T00:43:04.373 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2024-04-25T00:43:04.373 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T00:43:04.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 Processor -- start 2024-04-25T00:43:04.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 -- start start 2024-04-25T00:43:04.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fae101437b0 0x7fae100b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fae100b2b10 0x7fae100710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae10143520 con 0x7fae10143140 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fae100b2f40 con 0x7fae100b2b10 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae15809640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fae10070be0 con 0x7fae101437b0 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0effd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fae10143140 0x7fae1006cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46670/0 (socket says 172.21.15.129:46670) 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0effd640 1 -- 172.21.15.129:0/2741188298 learned_addr learned my addr 172.21.15.129:0/2741188298 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0e7fc640 1 --2- 172.21.15.129:0/2741188298 >> v2:172.21.15.196:3300/0 conn(0x7fae101437b0 0x7fae100b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0f7fe640 1 --2- 172.21.15.129:0/2741188298 >> v2:172.21.15.129:3300/0 conn(0x7fae100b2b10 0x7fae100710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0e7fc640 1 -- 172.21.15.129:0/2741188298 >> v2:172.21.15.129:3300/0 conn(0x7fae100b2b10 msgr2=0x7fae100710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0e7fc640 1 --2- 172.21.15.129:0/2741188298 >> v2:172.21.15.129:3300/0 conn(0x7fae100b2b10 0x7fae100710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0e7fc640 1 -- 172.21.15.129:0/2741188298 >> v1:172.21.15.129:6789/0 conn(0x7fae10143140 legacy=0x7fae1006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0e7fc640 1 -- 172.21.15.129:0/2741188298 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1006d400 con 0x7fae101437b0 2024-04-25T00:43:04.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.508+0000 7fae0f7fe640 1 --2- 172.21.15.129:0/2741188298 >> v2:172.21.15.129:3300/0 conn(0x7fae100b2b10 0x7fae100710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:43:04.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.512+0000 7fae0e7fc640 1 --2- 172.21.15.129:0/2741188298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae100b25d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae1006fe70 tx=0x7fae0801dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52b69f0f5c6afdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.512+0000 7fae0dffb640 1 -- 172.21.15.129:0/2741188298 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080a6cd0 con 0x7fae101437b0 2024-04-25T00:43:04.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.512+0000 7fae0dffb640 1 -- 172.21.15.129:0/2741188298 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0800a0d0 con 0x7fae101437b0 2024-04-25T00:43:04.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 -- 172.21.15.129:0/2741188298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 msgr2=0x7fae100b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 --2- 172.21.15.129:0/2741188298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae100b25d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae1006fe70 tx=0x7fae0801dc90 comp rx=0 tx=0).stop 2024-04-25T00:43:04.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 -- 172.21.15.129:0/2741188298 shutdown_connections 2024-04-25T00:43:04.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 --2- 172.21.15.129:0/2741188298 >> v2:172.21.15.129:3300/0 conn(0x7fae100b2b10 0x7fae100710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 --2- 172.21.15.129:0/2741188298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae100b25d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 -- 172.21.15.129:0/2741188298 >> 172.21.15.129:0/2741188298 conn(0x7fae10072d80 msgr2=0x7fae10073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 -- 172.21.15.129:0/2741188298 shutdown_connections 2024-04-25T00:43:04.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 -- 172.21.15.129:0/2741188298 wait complete. 2024-04-25T00:43:04.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 Processor -- start 2024-04-25T00:43:04.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.516+0000 7fae15809640 1 -- start start 2024-04-25T00:43:04.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae15809640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 0x7fae10162d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae15809640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae10143140 0x7fae1014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae15809640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae10152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae15809640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae10071fb0 con 0x7fae100b2b10 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae15809640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae10071e30 con 0x7fae101437b0 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae15809640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae10072130 con 0x7fae10143140 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 0x7fae10162d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 0x7fae10162d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42814/0 (socket says 172.21.15.129:42814) 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 -- 172.21.15.129:0/3428101403 learned_addr learned my addr 172.21.15.129:0/3428101403 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0e7fc640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae10143140 0x7fae1014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0f7fe640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae10152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 -- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae10143140 msgr2=0x7fae1014fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae10143140 0x7fae1014fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 -- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 msgr2=0x7fae10152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae10152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 -- 172.21.15.129:0/3428101403 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1014fff0 con 0x7fae100b2b10 2024-04-25T00:43:04.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.520+0000 7fae0effd640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 0x7fae10162d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c7b0 tx=0x7fae0000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00018070 con 0x7fae100b2b10 2024-04-25T00:43:04.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fae15809640 1 -- 172.21.15.129:0/3428101403 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae10163380 con 0x7fae100b2b10 2024-04-25T00:43:04.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fae15809640 1 -- 172.21.15.129:0/3428101403 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae10163940 con 0x7fae100b2b10 2024-04-25T00:43:04.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae000074e0 con 0x7fae100b2b10 2024-04-25T00:43:04.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0000ce60 con 0x7fae100b2b10 2024-04-25T00:43:04.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae0001b070 con 0x7fae100b2b10 2024-04-25T00:43:04.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fadf7fff640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fadf8067a50 0x7fadf8069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.524+0000 7fae0e7fc640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fadf8067a50 0x7fadf8069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.528+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae00085150 con 0x7fae100b2b10 2024-04-25T00:43:04.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.528+0000 7fae0e7fc640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fadf8067a50 0x7fadf8069f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae080af000 tx=0x7fae080a8070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.536+0000 7fae15809640 1 -- 172.21.15.129:0/3428101403 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf00052a0 con 0x7fae100b2b10 2024-04-25T00:43:04.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.544+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fae1014fff0 con 0x7fae100b2b10 2024-04-25T00:43:04.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.572+0000 7fbe6490a640 1 Processor -- start 2024-04-25T00:43:04.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.572+0000 7fbe6490a640 1 -- start start 2024-04-25T00:43:04.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.572+0000 7fbe6490a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60143140 0x7fbe6006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe6490a640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbe600b2ce0 0x7fbe600710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe6490a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe60143520 con 0x7fbe601437b0 2024-04-25T00:43:04.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe6490a640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fbe60070be0 con 0x7fbe600b2ce0 2024-04-25T00:43:04.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe6490a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6006d400 con 0x7fbe60143140 2024-04-25T00:43:04.579 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbe600b2ce0 0x7fbe600710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbe600b2ce0 0x7fbe600710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42816/0 (socket says 172.21.15.129:42816) 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/3844192066 learned_addr learned my addr 172.21.15.129:0/3844192066 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5e59c640 1 --2- 172.21.15.129:0/3844192066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60143140 0x7fbe6006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/3844192066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60143140 msgr2=0x7fbe6006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 --2- 172.21.15.129:0/3844192066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60143140 0x7fbe6006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/3844192066 >> v1:172.21.15.196:6789/0 conn(0x7fbe601437b0 legacy=0x7fbe600b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/3844192066 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6006d6e0 con 0x7fbe600b2ce0 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 Processor -- start 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 -- start start 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c0e33a0 0x7fb70c03a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb70c006260 0x7fb70c0525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70c04fdc0 con 0x7fb70c0506d0 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb70c052b20 con 0x7fb70c006260 2024-04-25T00:43:04.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71359e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c0500a0 con 0x7fb70c0e33a0 2024-04-25T00:43:04.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb70c006260 0x7fb70c0525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb70c006260 0x7fb70c0525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:45996/0 (socket says 172.21.15.129:45996) 2024-04-25T00:43:04.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb71259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c0e33a0 0x7fb70c03a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 -- 172.21.15.129:0/2613512277 learned_addr learned my addr 172.21.15.129:0/2613512277 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 -- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c0e33a0 msgr2=0x7fb70c03a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 --2- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c0e33a0 0x7fb70c03a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 -- 172.21.15.129:0/2613512277 >> v1:172.21.15.196:6789/0 conn(0x7fb70c0506d0 legacy=0x7fb70c039690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.576+0000 7fb711d9b640 1 -- 172.21.15.129:0/2613512277 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c038e90 con 0x7fb70c006260 2024-04-25T00:43:04.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe5ed9d640 1 --2- 172.21.15.129:0/3844192066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe600b2ce0 0x7fbe600710e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009870 tx=0x7fbe5401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9584b2c09fd6920e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 Processor -- start 2024-04-25T00:43:04.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 -- start start 2024-04-25T00:43:04.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 Processor -- start 2024-04-25T00:43:04.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 -- start start 2024-04-25T00:43:04.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe5d59a640 1 -- 172.21.15.129:0/3844192066 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540ab620 con 0x7fbe600b2ce0 2024-04-25T00:43:04.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe5d59a640 1 -- 172.21.15.129:0/3844192066 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540b0070 con 0x7fbe600b2ce0 2024-04-25T00:43:04.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff11015ae50 0x7ff11015ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1100af0a0 con 0x7ff1100cbe80 2024-04-25T00:43:04.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff110071c10 con 0x7ff11015ae50 2024-04-25T00:43:04.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff114824640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff110071ef0 con 0x7ff11015ef90 2024-04-25T00:43:04.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f71540e30d0 0x7f715403a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154053970 0x7f715404f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7154053e70 con 0x7f715404fd50 2024-04-25T00:43:04.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f7154037bc0 con 0x7f71540e30d0 2024-04-25T00:43:04.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715b59e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f71540501b0 con 0x7f7154053970 2024-04-25T00:43:04.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe5d59a640 1 -- 172.21.15.129:0/3844192066 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540ab9e0 con 0x7fbe600b2ce0 2024-04-25T00:43:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f71540e30d0 0x7f715403a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f71540e30d0 0x7f715403a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42818/0 (socket says 172.21.15.129:42818) 2024-04-25T00:43:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 -- 172.21.15.129:0/207253364 learned_addr learned my addr 172.21.15.129:0/207253364 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10ffff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff1100cbe80 0x7ff110070bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:54602/0 (socket says 172.21.15.129:54602) 2024-04-25T00:43:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10ffff640 1 -- 172.21.15.129:0/1620384142 learned_addr learned my addr 172.21.15.129:0/1620384142 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fc5a2290640 1 Processor -- start 2024-04-25T00:43:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f7159d9b640 1 --2- 172.21.15.129:0/207253364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154053970 0x7f715404f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10f7fe640 1 --2- 172.21.15.129:0/1620384142 >> v2:172.21.15.196:3300/0 conn(0x7ff11015ae50 0x7ff11015ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fc5a2290640 1 -- start start 2024-04-25T00:43:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe6490a640 1 -- 172.21.15.129:0/3844192066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe600b2ce0 msgr2=0x7fbe600710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe6490a640 1 --2- 172.21.15.129:0/3844192066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe600b2ce0 0x7fbe600710e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009870 tx=0x7fbe5401c530 comp rx=0 tx=0).stop 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe6490a640 1 -- 172.21.15.129:0/3844192066 shutdown_connections 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe6490a640 1 --2- 172.21.15.129:0/3844192066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe600b2ce0 0x7fbe600710e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe6490a640 1 --2- 172.21.15.129:0/3844192066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe60143140 0x7fbe6006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fbe6490a640 1 -- 172.21.15.129:0/3844192066 >> 172.21.15.129:0/3844192066 conn(0x7fbe60072d80 msgr2=0x7fbe60073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10effd640 1 --2- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10f7fe640 1 -- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 msgr2=0x7ff1100b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10f7fe640 1 --2- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10f7fe640 1 -- 172.21.15.129:0/1620384142 >> v1:172.21.15.196:6789/0 conn(0x7ff1100cbe80 legacy=0x7ff110070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7ff10f7fe640 1 -- 172.21.15.129:0/1620384142 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100721d0 con 0x7ff11015ae50 2024-04-25T00:43:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fb711d9b640 1 --2- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c006260 0x7fb70c0525e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb714050eb0 tx=0x7fb7140979c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8321754406b003b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 -- 172.21.15.129:0/207253364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154053970 msgr2=0x7f715404f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 --2- 172.21.15.129:0/207253364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154053970 0x7f715404f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 -- 172.21.15.129:0/207253364 >> v1:172.21.15.129:6789/0 conn(0x7f715404fd50 legacy=0x7f71540e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7f715a59c640 1 -- 172.21.15.129:0/207253364 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7154054150 con 0x7f71540e30d0 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fc5a2290640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 0x7fc59c163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fc5a2290640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c0cc780 con 0x7fc59c15ff90 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fc5a2290640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c0cca60 con 0x7fc59c0b2de0 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fc5a2290640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc59c0cd390 con 0x7fc59c160370 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fb71159a640 1 -- 172.21.15.129:0/2613512277 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71412c070 con 0x7fb70c006260 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fb71159a640 1 -- 172.21.15.129:0/2613512277 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714127600 con 0x7fb70c006260 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.580+0000 7fb71159a640 1 -- 172.21.15.129:0/2613512277 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714127a30 con 0x7fb70c006260 2024-04-25T00:43:04.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c006260 msgr2=0x7fb70c0525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 --2- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c006260 0x7fb70c0525e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb714050eb0 tx=0x7fb7140979c0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- 172.21.15.129:0/2613512277 shutdown_connections 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 --2- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c006260 0x7fb70c0525e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 --2- 172.21.15.129:0/2613512277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c0e33a0 0x7fb70c03a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- 172.21.15.129:0/2613512277 >> 172.21.15.129:0/2613512277 conn(0x7fb70c009f90 msgr2=0x7fb70c00a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fc5a0a8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 0x7fc59c163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fc5a0a8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 0x7fc59c163640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53666/0 (socket says 172.21.15.129:53666) 2024-04-25T00:43:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fc5a0a8d640 1 -- 172.21.15.129:0/285706253 learned_addr learned my addr 172.21.15.129:0/285706253 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- 172.21.15.129:0/2613512277 shutdown_connections 2024-04-25T00:43:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- 172.21.15.129:0/2613512277 wait complete. 2024-04-25T00:43:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fc5a0a8d640 1 -- 172.21.15.129:0/285706253 >> v1:172.21.15.196:6789/0 conn(0x7fc59c0b2de0 legacy=0x7fc59c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fc5a0a8d640 1 -- 172.21.15.129:0/285706253 >> v1:172.21.15.129:6789/0 conn(0x7fc59c15ff90 legacy=0x7fc59c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fc5a0a8d640 1 -- 172.21.15.129:0/285706253 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c0ccd40 con 0x7fc59c160370 2024-04-25T00:43:04.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fbe6490a640 1 -- 172.21.15.129:0/3844192066 shutdown_connections 2024-04-25T00:43:04.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fbe6490a640 1 -- 172.21.15.129:0/3844192066 wait complete. 2024-04-25T00:43:04.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7f715a59c640 1 --2- 172.21.15.129:0/207253364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71540e30d0 0x7f715403a440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f714c009870 tx=0x7f714c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1872387103e67e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 Processor -- start 2024-04-25T00:43:04.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7ff10f7fe640 1 --2- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff11015ea50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff100009820 tx=0x7ff1000208c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0197f3073b5e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7f715959a640 1 -- 172.21.15.129:0/207253364 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c0b0070 con 0x7f71540e30d0 2024-04-25T00:43:04.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7f715959a640 1 -- 172.21.15.129:0/207253364 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714c0ab600 con 0x7f71540e30d0 2024-04-25T00:43:04.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7f715959a640 1 -- 172.21.15.129:0/207253364 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c0ab9d0 con 0x7f71540e30d0 2024-04-25T00:43:04.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- start start 2024-04-25T00:43:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7ff10e7fc640 1 -- 172.21.15.129:0/1620384142 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b6070 con 0x7ff11015ae50 2024-04-25T00:43:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7ff10e7fc640 1 -- 172.21.15.129:0/1620384142 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1000a9770 con 0x7ff11015ae50 2024-04-25T00:43:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7f715b59e640 1 -- 172.21.15.129:0/207253364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71540e30d0 msgr2=0x7f715403a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7f715b59e640 1 --2- 172.21.15.129:0/207253364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71540e30d0 0x7f715403a440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f714c009870 tx=0x7f714c01c520 comp rx=0 tx=0).stop 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c006260 0x7fb70c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0506d0 0x7fb70c0dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb70c0e33a0 0x7fb70c0dc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c0e3780 con 0x7fb70c0e33a0 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c050380 con 0x7fb70c0506d0 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fb71359e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c050ef0 con 0x7fb70c006260 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7ff10e7fc640 1 -- 172.21.15.129:0/1620384142 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000a9b40 con 0x7ff11015ae50 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7ff114824640 1 -- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 msgr2=0x7ff11015ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7ff114824640 1 --2- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff11015ea50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff100009820 tx=0x7ff1000208c0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.584+0000 7fbe6490a640 1 Processor -- start 2024-04-25T00:43:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 Processor -- start 2024-04-25T00:43:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- 172.21.15.129:0/1620384142 shutdown_connections 2024-04-25T00:43:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 --2- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb70c0e33a0 0x7fb70c0dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb711d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0506d0 0x7fb70c0dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb71259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c006260 0x7fb70c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 --2- 172.21.15.129:0/1620384142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff11015ea50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- 172.21.15.129:0/1620384142 >> 172.21.15.129:0/1620384142 conn(0x7ff110067470 msgr2=0x7ff1100d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb71259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c006260 0x7fb70c0cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53680/0 (socket says 172.21.15.129:53680) 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb711d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0506d0 0x7fb70c0dbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:46020/0 (socket says 172.21.15.129:46020) 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb71259c640 1 -- 172.21.15.129:0/3842235006 learned_addr learned my addr 172.21.15.129:0/3842235006 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 -- start start 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- 172.21.15.129:0/207253364 shutdown_connections 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 --2- 172.21.15.129:0/207253364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154053970 0x7f715404f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 --2- 172.21.15.129:0/207253364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71540e30d0 0x7f715403a440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- 172.21.15.129:0/207253364 >> 172.21.15.129:0/207253364 conn(0x7f7154009fc0 msgr2=0x7f715400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- 172.21.15.129:0/1620384142 shutdown_connections 2024-04-25T00:43:04.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- 172.21.15.129:0/207253364 shutdown_connections 2024-04-25T00:43:04.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc5a0a8d640 1 --2- 172.21.15.129:0/285706253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 0x7fc59c163640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5980098a0 tx=0x7fc59801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfdad5ae70c5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb93e90a640 1 Processor -- start 2024-04-25T00:43:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 -- 172.21.15.129:0/3842235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c006260 msgr2=0x7fb70c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c006260 0x7fb70c0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 -- 172.21.15.129:0/3842235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0506d0 msgr2=0x7fb70c0dbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0506d0 0x7fb70c0dbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 -- 172.21.15.129:0/3842235006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c048db0 con 0x7fb70c0e33a0 2024-04-25T00:43:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- 172.21.15.129:0/1620384142 wait complete. 2024-04-25T00:43:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb712d9d640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb70c0e33a0 0x7fb70c0dc490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb70400c3c0 tx=0x7fb70400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc593fff640 1 -- 172.21.15.129:0/285706253 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5980ab650 con 0x7fc59c160370 2024-04-25T00:43:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc593fff640 1 -- 172.21.15.129:0/285706253 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5980b0070 con 0x7fc59c160370 2024-04-25T00:43:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc593fff640 1 -- 172.21.15.129:0/285706253 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5980aba10 con 0x7fc59c160370 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 -- start start 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600b2ce0 0x7fbe60156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe60143140 0x7fbe60140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 0x7fbe601409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60071fb0 con 0x7fbe601437b0 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60071e30 con 0x7fbe60143140 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe60072130 con 0x7fbe600b2ce0 2024-04-25T00:43:04.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- 172.21.15.129:0/207253364 wait complete. 2024-04-25T00:43:04.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 0x7fbe601409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 0x7fbe601409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42828/0 (socket says 172.21.15.129:42828) 2024-04-25T00:43:04.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/591504194 learned_addr learned my addr 172.21.15.129:0/591504194 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7840cf7a0 0x7fa7840ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa78415f100 0x7fa7840cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa78406dfd0 con 0x7fa78406fbe0 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa78406ee60 con 0x7fa78415f100 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78359e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa78406e250 con 0x7fa7840cf7a0 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7840cf7a0 0x7fa7840ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa781d9b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fa78406fbe0 0x7fa78415e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46692/0 (socket says 172.21.15.129:46692) 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa781d9b640 1 -- 172.21.15.129:0/2609375747 learned_addr learned my addr 172.21.15.129:0/2609375747 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc5a2290640 1 -- 172.21.15.129:0/285706253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 msgr2=0x7fc59c163640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc5a2290640 1 --2- 172.21.15.129:0/285706253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 0x7fc59c163640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5980098a0 tx=0x7fc59801c560 comp rx=0 tx=0).stop 2024-04-25T00:43:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 Processor -- start 2024-04-25T00:43:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 Processor -- start 2024-04-25T00:43:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa782d9d640 1 --2- 172.21.15.129:0/2609375747 >> v2:172.21.15.196:3300/0 conn(0x7fa78415f100 0x7fa7840cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/591504194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600b2ce0 msgr2=0x7fbe60156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600b2ce0 0x7fbe60156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/591504194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe60143140 msgr2=0x7fbe60140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe60143140 0x7fbe60140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 -- 172.21.15.129:0/591504194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6013cc70 con 0x7fbe601437b0 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe5ed9d640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 0x7fbe601409a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe5401abe0 tx=0x7fbe5401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- start start 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7040052d0 con 0x7fb70c0e33a0 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704005490 con 0x7fb70c0e33a0 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704007240 con 0x7fb70c0e33a0 2024-04-25T00:43:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c049090 con 0x7fb70c0e33a0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c049650 con 0x7fb70c0e33a0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540abbd0 con 0x7fbe601437b0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540abd90 con 0x7fbe601437b0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6013cf50 con 0x7fbe601437b0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6013d510 con 0x7fbe601437b0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540be630 con 0x7fbe601437b0 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- start start 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f715404fd50 0x7f71540ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7154053970 0x7f71540e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71540e30d0 0x7f715417c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71540e3630 con 0x7f7154053970 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71540e34b0 con 0x7f71540e30d0 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715b59e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f71540f0030 con 0x7f715404fd50 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1100cbe80 0x7ff110151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff1100ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100b2d90 con 0x7ff1100cbe80 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100b2c10 con 0x7ff11015ae50 2024-04-25T00:43:04.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff114824640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1100b2f10 con 0x7ff11015ef90 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424632106 0 0) 0x7fa774001120 con 0x7fa78406fbe0 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa76c003530 con 0x7fa78406fbe0 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1167571997 0 0) 0x7fa76c003530 con 0x7fa78406fbe0 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7840cf7a0 msgr2=0x7fa7840ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7840cf7a0 0x7fa7840ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 >> v2:172.21.15.196:3300/0 conn(0x7fa78415f100 msgr2=0x7fa7840cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 --2- 172.21.15.129:0/2609375747 >> v2:172.21.15.196:3300/0 conn(0x7fa78415f100 0x7fa7840cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78406e530 con 0x7fa78406fbe0 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fa774001850 con 0x7fa78406fbe0 2024-04-25T00:43:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 >> v1:172.21.15.129:6789/0 conn(0x7fa78406fbe0 legacy=0x7fa78415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa76c003a30 0x7fa76c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa76c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa76c005220 0x7fa76c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa78406ee60 con 0x7fa76c004810 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa78406e250 con 0x7fa76c003a30 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c00c150 con 0x7fa76c005220 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fa782d9d640 1 --2- 172.21.15.129:0/2609375747 >> v2:172.21.15.196:3300/0 conn(0x7fa78415f100 0x7fa7840cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f7159d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7154053970 0x7f71540e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f7159d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7154053970 0x7f71540e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42846/0 (socket says 172.21.15.129:42846) 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f7159d9b640 1 -- 172.21.15.129:0/794598706 learned_addr learned my addr 172.21.15.129:0/794598706 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7f715a59c640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f715404fd50 0x7f71540ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715ad9d640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71540e30d0 0x7f715417c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff10effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff1100ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff10effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff1100ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:46066/0 (socket says 172.21.15.129:46066) 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7ff10effd640 1 -- 172.21.15.129:0/2447227816 learned_addr learned my addr 172.21.15.129:0/2447227816 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10f7fe640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1100cbe80 0x7ff110151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10ffff640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa782d9d640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa76c005220 0x7fa76c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa78259c640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa76c003a30 0x7fa76c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c005370 con 0x7fb70c0e33a0 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb93e90a640 1 -- start start 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb93e90a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb93e90a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb938143520 con 0x7fb9381437b0 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb93e90a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb938070be0 con 0x7fb938143140 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb93e90a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb93806d400 con 0x7fb9380b2ce0 2024-04-25T00:43:04.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb937fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb938143140 0x7fb93806cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:54630/0 (socket says 172.21.15.129:54630) 2024-04-25T00:43:04.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fb937fff640 1 -- 172.21.15.129:0/4266934853 learned_addr learned my addr 172.21.15.129:0/4266934853 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:04.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc5a2290640 1 -- 172.21.15.129:0/285706253 shutdown_connections 2024-04-25T00:43:04.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc5a2290640 1 --2- 172.21.15.129:0/285706253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c160370 0x7fc59c163640 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.588+0000 7fc5a2290640 1 -- 172.21.15.129:0/285706253 >> 172.21.15.129:0/285706253 conn(0x7fc59c067470 msgr2=0x7fc59c0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe440052a0 con 0x7fbe601437b0 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715a59c640 1 -- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71540e30d0 msgr2=0x7f715417c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715a59c640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71540e30d0 0x7f715417c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715a59c640 1 -- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7154053970 msgr2=0x7f71540e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715a59c640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7154053970 0x7f71540e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715a59c640 1 -- 172.21.15.129:0/794598706 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71540ed6e0 con 0x7f715404fd50 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10effd640 1 -- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 msgr2=0x7ff1100ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10effd640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10effd640 1 -- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1100cbe80 msgr2=0x7ff110151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10effd640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1100cbe80 0x7ff110151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10effd640 1 -- 172.21.15.129:0/2447227816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100acf30 con 0x7ff11015ae50 2024-04-25T00:43:04.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fc5a2290640 1 -- 172.21.15.129:0/285706253 shutdown_connections 2024-04-25T00:43:04.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715a59c640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f715404fd50 0x7f71540ed1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f714c0ab000 tx=0x7f714c0192c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff10effd640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff1100ac3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff10400d7d0 tx=0x7ff10400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7040073e0 con 0x7fb70c0e33a0 2024-04-25T00:43:04.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb702ffd640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6f40679c0 0x7fb6f4069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb704084ef0 con 0x7fb70c0e33a0 2024-04-25T00:43:04.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb71259c640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6f40679c0 0x7fb6f4069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400b6c0 con 0x7ff11015ae50 2024-04-25T00:43:04.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff104019070 con 0x7ff11015ae50 2024-04-25T00:43:04.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1100a9620 con 0x7ff11015ae50 2024-04-25T00:43:04.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb71259c640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6f40679c0 0x7fb6f4069e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0e1520 tx=0x7fb708009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1100a9be0 con 0x7ff11015ae50 2024-04-25T00:43:04.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400baa0 con 0x7ff11015ae50 2024-04-25T00:43:04.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbe540b43f0 con 0x7fbe601437b0 2024-04-25T00:43:04.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fbe4effd640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe4006bd20 0x7fbe4006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe5411fc40 con 0x7fbe601437b0 2024-04-25T00:43:04.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c0b9020 con 0x7f715404fd50 2024-04-25T00:43:04.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714c0b1040 con 0x7f715404fd50 2024-04-25T00:43:04.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c0b0050 con 0x7f715404fd50 2024-04-25T00:43:04.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71540e5bf0 con 0x7f715404fd50 2024-04-25T00:43:04.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f00052a0 con 0x7ff11015ae50 2024-04-25T00:43:04.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa76c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa781d9b640 1 -- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa76c005220 msgr2=0x7fa76c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa76c005220 0x7fa76c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa781d9b640 1 -- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa76c003a30 msgr2=0x7fa76c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa76c003a30 0x7fa76c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fa781d9b640 1 -- 172.21.15.129:0/2609375747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78406e530 con 0x7fa76c004810 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f715404c7a0 con 0x7f715404fd50 2024-04-25T00:43:04.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71540053a0 con 0x7f715404fd50 2024-04-25T00:43:04.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 -- 172.21.15.129:0/285706253 wait complete. 2024-04-25T00:43:04.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa76c004c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa774001dc0 tx=0x7fa774028100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf141d3286ebd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff10400bc40 con 0x7ff11015ae50 2024-04-25T00:43:04.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7ff0f7fff640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0fc067910 0x7ff0fc069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff104086bd0 con 0x7ff11015ae50 2024-04-25T00:43:04.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 Processor -- start 2024-04-25T00:43:04.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fbe5e59c640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe4006bd20 0x7fbe4006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7ff10f7fe640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0fc067910 0x7ff0fc069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7ff10f7fe640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0fc067910 0x7ff0fc069dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b3000 tx=0x7ff100008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 -- start start 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c0b2de0 0x7fc59c1457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59c15ff90 0x7fc59c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 0x7fc59c15e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c0ce220 con 0x7fc59c160370 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c0ce0a0 con 0x7fc59c15ff90 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a2290640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc59c170560 con 0x7fc59c0b2de0 2024-04-25T00:43:04.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 0x7fc59c15e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 0x7fc59c15e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42862/0 (socket says 172.21.15.129:42862) 2024-04-25T00:43:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 -- 172.21.15.129:0/1755715522 learned_addr learned my addr 172.21.15.129:0/1755715522 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a128e640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c0b2de0 0x7fc59c1457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a0a8d640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59c15ff90 0x7fc59c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f714c0bc020 con 0x7f715404fd50 2024-04-25T00:43:04.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7f714affd640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7144067ad0 0x7f7144069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 -- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c0b2de0 msgr2=0x7fc59c1457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c0b2de0 0x7fc59c1457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 -- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59c15ff90 msgr2=0x7fc59c06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59c15ff90 0x7fc59c06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 -- 172.21.15.129:0/1755715522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c146130 con 0x7fc59c160370 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7f7159d9b640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7144067ad0 0x7f7144069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f714c11ffe0 con 0x7f715404fd50 2024-04-25T00:43:04.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fbe5e59c640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe4006bd20 0x7fbe4006e1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe60159230 tx=0x7fbe5800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.596+0000 7fc5a1a8f640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 0x7fc59c15e980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c3c0 tx=0x7fc58c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.600+0000 7f7159d9b640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7144067ad0 0x7f7144069f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f715c050eb0 tx=0x7f715c08e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb936ffd640 1 -- 172.21.15.129:0/4266934853 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668801300 0 0) 0x7fb938070be0 con 0x7fb938143140 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.592+0000 7fb936ffd640 1 -- 172.21.15.129:0/4266934853 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb920003530 con 0x7fb938143140 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.600+0000 7fb93cea7640 1 --2- 172.21.15.129:0/4266934853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.600+0000 7fb93cea7640 1 -- 172.21.15.129:0/4266934853 >> v1:172.21.15.196:6789/0 conn(0x7fb938143140 legacy=0x7fb93806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.600+0000 7fb93cea7640 1 -- 172.21.15.129:0/4266934853 >> v1:172.21.15.129:6789/0 conn(0x7fb9381437b0 legacy=0x7fb9380b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.600+0000 7fb93cea7640 1 -- 172.21.15.129:0/4266934853 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93806d6e0 con 0x7fb9380b2ce0 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.604+0000 7fb93cea7640 1 --2- 172.21.15.129:0/4266934853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938164db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb92800dc10 tx=0x7fb928024bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676c3540c4891879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.604+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb704055450 con 0x7fb70c0e33a0 2024-04-25T00:43:04.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.604+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0052d0 con 0x7fc59c160370 2024-04-25T00:43:04.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.604+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c146410 con 0x7fc59c160370 2024-04-25T00:43:04.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.604+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1469d0 con 0x7fc59c160370 2024-04-25T00:43:04.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.604+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c146ec0 con 0x7fc59c160370 2024-04-25T00:43:04.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.608+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff104086e20 con 0x7ff11015ae50 2024-04-25T00:43:04.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb936ffd640 1 -- 172.21.15.129:0/4266934853 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92800d2d0 con 0x7fb9380b2ce0 2024-04-25T00:43:04.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb936ffd640 1 -- 172.21.15.129:0/4266934853 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9280adaa0 con 0x7fb9380b2ce0 2024-04-25T00:43:04.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f714c016110 con 0x7f715404fd50 2024-04-25T00:43:04.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb936ffd640 1 -- 172.21.15.129:0/4266934853 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280ade50 con 0x7fb9380b2ce0 2024-04-25T00:43:04.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 -- 172.21.15.129:0/4266934853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 msgr2=0x7fb938164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 --2- 172.21.15.129:0/4266934853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938164db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb92800dc10 tx=0x7fb928024bd0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 -- 172.21.15.129:0/4266934853 shutdown_connections 2024-04-25T00:43:04.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 --2- 172.21.15.129:0/4266934853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938164db0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 -- 172.21.15.129:0/4266934853 >> 172.21.15.129:0/4266934853 conn(0x7fb938072d80 msgr2=0x7fb938073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 -- 172.21.15.129:0/4266934853 shutdown_connections 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 -- 172.21.15.129:0/4266934853 wait complete. 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 Processor -- start 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.612+0000 7fb93e90a640 1 -- start start 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb93e90a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb93e90a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb938143140 0x7fb938138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77400e310 con 0x7fa76c004810 2024-04-25T00:43:04.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbe54016150 con 0x7fbe601437b0 2024-04-25T00:43:04.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7740b5020 con 0x7fa76c004810 2024-04-25T00:43:04.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb937fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb937fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53740/0 (socket says 172.21.15.129:53740) 2024-04-25T00:43:04.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb937fff640 1 -- 172.21.15.129:0/700583418 learned_addr learned my addr 172.21.15.129:0/700583418 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb9377fe640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb938143140 0x7fb938138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c005470 con 0x7fc59c160370 2024-04-25T00:43:04.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c020060 con 0x7fc59c160370 2024-04-25T00:43:04.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78159a640 1 -- 172.21.15.129:0/2609375747 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740b1e50 con 0x7fa76c004810 2024-04-25T00:43:04.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9381437b0 0x7fb938135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb938167ea0 con 0x7fb9381437b0 2024-04-25T00:43:04.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb938167d20 con 0x7fb938143140 2024-04-25T00:43:04.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb938168020 con 0x7fb9380b2ce0 2024-04-25T00:43:04.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78359e640 1 -- 172.21.15.129:0/2609375747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 msgr2=0x7fa76c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78359e640 1 --2- 172.21.15.129:0/2609375747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa76c004c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa774001dc0 tx=0x7fa774028100 comp rx=0 tx=0).stop 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa782d9d640 1 -- 172.21.15.129:0/2609375747 reap_dead start 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78359e640 1 -- 172.21.15.129:0/2609375747 shutdown_connections 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78359e640 1 -- 172.21.15.129:0/2609375747 >> 172.21.15.129:0/2609375747 conn(0x7fa784067070 msgr2=0x7fa784067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb9377fe640 1 -- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 msgr2=0x7fb938159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb9377fe640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb9377fe640 1 -- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9381437b0 msgr2=0x7fb938135490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb9377fe640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9381437b0 0x7fb938135490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fb9377fe640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb938139910 con 0x7fb938143140 2024-04-25T00:43:04.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78359e640 1 -- 172.21.15.129:0/2609375747 shutdown_connections 2024-04-25T00:43:04.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fa78359e640 1 -- 172.21.15.129:0/2609375747 wait complete. 2024-04-25T00:43:04.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.616+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc58c007260 con 0x7fc59c160370 2024-04-25T00:43:04.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fc591ffb640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc5800678c0 0x7fc580069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc58c085310 con 0x7fc59c160370 2024-04-25T00:43:04.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fc5a128e640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc5800678c0 0x7fc580069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb9377fe640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb938143140 0x7fb938138e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c390 tx=0x7fb92c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c005220 con 0x7fb938143140 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c0053c0 con 0x7fb938143140 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c007230 con 0x7fb938143140 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb938139bf0 con 0x7fb938143140 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93813a1b0 con 0x7fb938143140 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fa78359e640 1 Processor -- start 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fc5a128e640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc5800678c0 0x7fc580069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc59c1411a0 tx=0x7fc594009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc58c055870 con 0x7fc59c160370 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fa78359e640 1 -- start start 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa784155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa784140d00 0x7fa784159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa784159d90 0x7fa784157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa78406f2f0 con 0x7fa76c004810 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa78406eaf0 con 0x7fa784159d90 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7840cfb80 con 0x7fa784140d00 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa784155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa784155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42874/0 (socket says 172.21.15.129:42874) 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 -- 172.21.15.129:0/2136741784 learned_addr learned my addr 172.21.15.129:0/2136741784 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:04.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa784140d00 0x7fa784159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa782d9d640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa784159d90 0x7fa784157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 -- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa784140d00 msgr2=0x7fa784159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa784140d00 0x7fa784159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 -- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa784159d90 msgr2=0x7fa784157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa784159d90 0x7fa784157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 -- 172.21.15.129:0/2136741784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa784141180 con 0x7fa76c004810 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78259c640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa784155a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa778010830 tx=0x7fa778010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77803a020 con 0x7fa76c004810 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa778004240 con 0x7fa76c004810 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77800e750 con 0x7fa76c004810 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 -- 172.21.15.129:0/2136741784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78415a2c0 con 0x7fa76c004810 2024-04-25T00:43:04.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fa78359e640 1 -- 172.21.15.129:0/2136741784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78415a880 con 0x7fa76c004810 2024-04-25T00:43:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.620+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb938062e50 con 0x7fb938143140 2024-04-25T00:43:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb92c019020 con 0x7fb938143140 2024-04-25T00:43:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fb934ff9640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9240678c0 0x7fb924069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb92c084d60 con 0x7fb938143140 2024-04-25T00:43:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fb937fff640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9240678c0 0x7fb924069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.624+0000 7fb937fff640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9240678c0 0x7fb924069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb930001570 tx=0x7fb930009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.628+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75c0052a0 con 0x7fa76c004810 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.632+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa77803d020 con 0x7fa76c004810 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.632+0000 7fa772ffd640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa7640678c0 0x7fa764069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.632+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa7780a8bf0 con 0x7fa76c004810 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.632+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa7640678c0 0x7fa764069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.632+0000 7fa781d9b640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa7640678c0 0x7fa764069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa78406ffc0 tx=0x7fa774001bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.636+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb92c055930 con 0x7fb938143140 2024-04-25T00:43:04.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.640+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa778044020 con 0x7fa76c004810 2024-04-25T00:43:04.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.848+0000 7fae15809640 1 -- 172.21.15.129:0/3428101403 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fadf0005860 con 0x7fae100b2b10 2024-04-25T00:43:04.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.852+0000 7fadf7fff640 1 -- 172.21.15.129:0/3428101403 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae00055720 con 0x7fae100b2b10 2024-04-25T00:43:04.855 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2024-04-25T00:43:04.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.852+0000 7fadf5ffb640 1 -- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fadf8067a50 msgr2=0x7fadf8069f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.852+0000 7fadf5ffb640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fadf8067a50 0x7fadf8069f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae080af000 tx=0x7fae080a8070 comp rx=0 tx=0).stop 2024-04-25T00:43:04.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 -- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 msgr2=0x7fae10162d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 0x7fae10162d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c7b0 tx=0x7fae0000cc70 comp rx=0 tx=0).stop 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 -- 172.21.15.129:0/3428101403 shutdown_connections 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fadf8067a50 0x7fadf8069f10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae10143140 0x7fae1014fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae101437b0 0x7fae10152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 --2- 172.21.15.129:0/3428101403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae100b2b10 0x7fae10162d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 -- 172.21.15.129:0/3428101403 >> 172.21.15.129:0/3428101403 conn(0x7fae10072d80 msgr2=0x7fae100a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 -- 172.21.15.129:0/3428101403 shutdown_connections 2024-04-25T00:43:04.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.856+0000 7fadf5ffb640 1 -- 172.21.15.129:0/3428101403 wait complete. 2024-04-25T00:43:04.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-25T00:43:04.883 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.916+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa75c005590 con 0x7fa76c004810 2024-04-25T00:43:04.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.916+0000 7fa772ffd640 1 -- 172.21.15.129:0/2136741784 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa778078620 con 0x7fa76c004810 2024-04-25T00:43:04.922 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa7640678c0 msgr2=0x7fa764069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa7640678c0 0x7fa764069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa78406ffc0 tx=0x7fa774001bc0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 msgr2=0x7fa784155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa784155a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa778010830 tx=0x7fa778010cf0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 shutdown_connections 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa7640678c0 0x7fa764069d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa784140d00 0x7fa784159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa784159d90 0x7fa784157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 --2- 172.21.15.129:0/2136741784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa76c004810 0x7fa784155a30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 >> 172.21.15.129:0/2136741784 conn(0x7fa784067070 msgr2=0x7fa7840cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 shutdown_connections 2024-04-25T00:43:04.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.920+0000 7fa770ff9640 1 -- 172.21.15.129:0/2136741784 wait complete. 2024-04-25T00:43:04.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.924+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb70c0ed670 con 0x7fb70c0e33a0 2024-04-25T00:43:04.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.924+0000 7fb702ffd640 1 -- 172.21.15.129:0/3842235006 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb704059750 con 0x7fb70c0e33a0 2024-04-25T00:43:04.929 INFO:teuthology.orchestra.run.smithi129.stdout:55834574856 2024-04-25T00:43:04.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6f40679c0 msgr2=0x7fb6f4069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6f40679c0 0x7fb6f4069e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0e1520 tx=0x7fb708009190 comp rx=0 tx=0).stop 2024-04-25T00:43:04.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb70c0e33a0 msgr2=0x7fb70c0dc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb70c0e33a0 0x7fb70c0dc490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb70400c3c0 tx=0x7fb70400c880 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 shutdown_connections 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6f40679c0 0x7fb6f4069e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb70c0e33a0 0x7fb70c0dc490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0506d0 0x7fb70c0dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 --2- 172.21.15.129:0/3842235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c006260 0x7fb70c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 >> 172.21.15.129:0/3842235006 conn(0x7fb70c009f90 msgr2=0x7fb70c04f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 shutdown_connections 2024-04-25T00:43:04.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.928+0000 7fb71359e640 1 -- 172.21.15.129:0/3842235006 wait complete. 2024-04-25T00:43:04.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.936+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbe44005860 con 0x7fbe601437b0 2024-04-25T00:43:04.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.936+0000 7fbe4effd640 1 -- 172.21.15.129:0/591504194 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbe540f0900 con 0x7fbe601437b0 2024-04-25T00:43:04.940 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2024-04-25T00:43:04.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-25T00:43:04.943 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe4006bd20 msgr2=0x7fbe4006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe4006bd20 0x7fbe4006e1e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe60159230 tx=0x7fbe5800b040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 msgr2=0x7fbe601409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 0x7fbe601409a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe5401abe0 tx=0x7fbe5401ad70 comp rx=0 tx=0).stop 2024-04-25T00:43:04.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 shutdown_connections 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe4006bd20 0x7fbe4006e1e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe601437b0 0x7fbe601409a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe60143140 0x7fbe60140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 --2- 172.21.15.129:0/591504194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe600b2ce0 0x7fbe60156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 >> 172.21.15.129:0/591504194 conn(0x7fbe60072d80 msgr2=0x7fbe600ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 shutdown_connections 2024-04-25T00:43:04.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fbe6490a640 1 -- 172.21.15.129:0/591504194 wait complete. 2024-04-25T00:43:04.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.940+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc59c075320 con 0x7fc59c160370 2024-04-25T00:43:04.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc591ffb640 1 -- 172.21.15.129:0/1755715522 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc58c059b70 con 0x7fc59c160370 2024-04-25T00:43:04.947 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc5800678c0 msgr2=0x7fc580069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc5800678c0 0x7fc580069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc59c1411a0 tx=0x7fc594009040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 msgr2=0x7fc59c15e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 0x7fc59c15e980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c3c0 tx=0x7fc58c00c880 comp rx=0 tx=0).stop 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 shutdown_connections 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc5800678c0 0x7fc580069d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc59c160370 0x7fc59c15e980 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59c15ff90 0x7fc59c06d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 --2- 172.21.15.129:0/1755715522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59c0b2de0 0x7fc59c1457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.944+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 >> 172.21.15.129:0/1755715522 conn(0x7fc59c067470 msgr2=0x7fc59c161c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.948+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 shutdown_connections 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.948+0000 7fc5a2290640 1 -- 172.21.15.129:0/1755715522 wait complete. 2024-04-25T00:43:04.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.948+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff0f0005860 con 0x7ff11015ae50 2024-04-25T00:43:04.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.948+0000 7ff0f7fff640 1 -- 172.21.15.129:0/2447227816 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff1040576f0 con 0x7ff11015ae50 2024-04-25T00:43:04.953 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f71540487a0 con 0x7f715404fd50 2024-04-25T00:43:04.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7f714affd640 1 -- 172.21.15.129:0/794598706 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f714c0f0b90 con 0x7f715404fd50 2024-04-25T00:43:04.955 INFO:teuthology.orchestra.run.smithi129.stdout:55834574854 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0fc067910 msgr2=0x7ff0fc069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0fc067910 0x7ff0fc069dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b3000 tx=0x7ff100008040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 msgr2=0x7ff1100ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff1100ac3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff10400d7d0 tx=0x7ff10400dc90 comp rx=0 tx=0).stop 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 shutdown_connections 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff0fc067910 0x7ff0fc069dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff11015ef90 0x7ff1100ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff11015ae50 0x7ff1100ac3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 --2- 172.21.15.129:0/2447227816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1100cbe80 0x7ff110151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 >> 172.21.15.129:0/2447227816 conn(0x7ff110067470 msgr2=0x7ff1100affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 shutdown_connections 2024-04-25T00:43:04.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.952+0000 7ff114824640 1 -- 172.21.15.129:0/2447227816 wait complete. 2024-04-25T00:43:04.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7144067ad0 msgr2=0x7f7144069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7144067ad0 0x7f7144069f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f715c050eb0 tx=0x7f715c08e040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f715404fd50 msgr2=0x7f71540ed1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f715404fd50 0x7f71540ed1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f714c0ab000 tx=0x7f714c0192c0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 shutdown_connections 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7144067ad0 0x7f7144069f90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71540e30d0 0x7f715417c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7154053970 0x7f71540e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 --2- 172.21.15.129:0/794598706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f715404fd50 0x7f71540ed1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 >> 172.21.15.129:0/794598706 conn(0x7f7154009fc0 msgr2=0x7f715403a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 shutdown_connections 2024-04-25T00:43:04.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.956+0000 7f715b59e640 1 -- 172.21.15.129:0/794598706 wait complete. 2024-04-25T00:43:04.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-04-25T00:43:04.962 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.960+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb9380cf390 con 0x7fb938143140 2024-04-25T00:43:04.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.960+0000 7fb934ff9640 1 -- 172.21.15.129:0/700583418 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb92c055230 con 0x7fb938143140 2024-04-25T00:43:04.964 INFO:teuthology.orchestra.run.smithi129.stdout:55834574855 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9240678c0 msgr2=0x7fb924069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9240678c0 0x7fb924069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb930001570 tx=0x7fb930009040 comp rx=0 tx=0).stop 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb938143140 msgr2=0x7fb938138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb938143140 0x7fb938138e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00c390 tx=0x7fb92c00c850 comp rx=0 tx=0).stop 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 shutdown_connections 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9240678c0 0x7fb924069d80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9381437b0 0x7fb938135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb938143140 0x7fb938138e50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 --2- 172.21.15.129:0/700583418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9380b2ce0 0x7fb938159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 >> 172.21.15.129:0/700583418 conn(0x7fb938072d80 msgr2=0x7fb938073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 shutdown_connections 2024-04-25T00:43:04.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:04.964+0000 7fb93e90a640 1 -- 172.21.15.129:0/700583418 wait complete. 2024-04-25T00:43:04.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-25T00:43:04.972 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-25T00:43:04.983 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-25T00:43:04.988 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-25T00:43:04.989 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T00:43:04.994 DEBUG:teuthology.parallel:result is None 2024-04-25T00:43:04.994 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:43:04.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:43:05.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 Processor -- start 2024-04-25T00:43:05.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 -- start start 2024-04-25T00:43:05.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f729406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f729406b960 0x7f7294140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7294072730 con 0x7f72940cd410 2024-04-25T00:43:05.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f729406bdd0 con 0x7f729406b960 2024-04-25T00:43:05.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f729ac36640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f729413eb70 con 0x7f729413e710 2024-04-25T00:43:05.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72989d2640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f72940cd410 0x7f7294072020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46722/0 (socket says 172.21.15.129:46722) 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72991d3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f729406b960 0x7f7294140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72989d2640 1 -- 172.21.15.129:0/702102657 learned_addr learned my addr 172.21.15.129:0/702102657 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f7293fff640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f729406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72991d3640 1 -- 172.21.15.129:0/702102657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 msgr2=0x7f729406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72991d3640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f729406b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72991d3640 1 -- 172.21.15.129:0/702102657 >> v1:172.21.15.129:6789/0 conn(0x7f72940cd410 legacy=0x7f7294072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72991d3640 1 -- 172.21.15.129:0/702102657 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729415b770 con 0x7f729406b960 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f7293fff640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f729406b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:05.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72937fe640 1 -- 172.21.15.129:0/702102657 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686221602 0 0) 0x7f7288001120 con 0x7f72940cd410 2024-04-25T00:43:05.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72991d3640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f7294140950 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f728c009d90 tx=0x7f728c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b67ed5b1a97e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.140+0000 7f72937fe640 1 -- 172.21.15.129:0/702102657 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0b2070 con 0x7f729406b960 2024-04-25T00:43:05.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72937fe640 1 -- 172.21.15.129:0/702102657 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728c0a5830 con 0x7f729406b960 2024-04-25T00:43:05.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72937fe640 1 -- 172.21.15.129:0/702102657 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0a5c00 con 0x7f729406b960 2024-04-25T00:43:05.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/702102657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 msgr2=0x7f7294140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f7294140950 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f728c009d90 tx=0x7f728c01c970 comp rx=0 tx=0).stop 2024-04-25T00:43:05.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/702102657 shutdown_connections 2024-04-25T00:43:05.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f7294140950 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 --2- 172.21.15.129:0/702102657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f729406b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/702102657 >> 172.21.15.129:0/702102657 conn(0x7f72940b2380 msgr2=0x7f72940b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/702102657 shutdown_connections 2024-04-25T00:43:05.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/702102657 wait complete. 2024-04-25T00:43:05.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 Processor -- start 2024-04-25T00:43:05.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- start start 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f729413d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72940cd410 0x7f72941f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f72940aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f729406c6d0 con 0x7f729406b960 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f729406c550 con 0x7f72940cd410 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f72940cd7f0 con 0x7f729413e710 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f729413d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f729413d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42886/0 (socket says 172.21.15.129:42886) 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 -- 172.21.15.129:0/96240247 learned_addr learned my addr 172.21.15.129:0/96240247 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:05.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72991d3640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f72940aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 -- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 msgr2=0x7f72940aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f72940aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 -- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72940cd410 msgr2=0x7f72941f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72940cd410 0x7f72941f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 -- 172.21.15.129:0/96240247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72941565a0 con 0x7f729406b960 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72989d2640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f729413d900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f728800c780 tx=0x7f728800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288018070 con 0x7f729406b960 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72880072f0 con 0x7f729406b960 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7294156880 con 0x7f729406b960 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72880047f0 con 0x7f729406b960 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.144+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7294156e40 con 0x7f729406b960 2024-04-25T00:43:05.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.148+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7294062e50 con 0x7f729406b960 2024-04-25T00:43:05.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.148+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f728801b070 con 0x7f729406b960 2024-04-25T00:43:05.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.148+0000 7f72917fa640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f72780678f0 0x7f7278069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.148+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7288084ff0 con 0x7f729406b960 2024-04-25T00:43:05.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.148+0000 7f7293fff640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f72780678f0 0x7f7278069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.148+0000 7f7293fff640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f72780678f0 0x7f7278069db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f729415b2c0 tx=0x7f7280008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.152+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7294072730 con 0x7f729406b960 2024-04-25T00:43:05.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.296+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7294075320 con 0x7f72780678f0 2024-04-25T00:43:05.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.304+0000 7f72917fa640 1 -- 172.21.15.129:0/96240247 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f7294075320 con 0x7f72780678f0 2024-04-25T00:43:05.311 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:05.312 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:43:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f72780678f0 msgr2=0x7f7278069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f72780678f0 0x7f7278069db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f729415b2c0 tx=0x7f7280008040 comp rx=0 tx=0).stop 2024-04-25T00:43:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 msgr2=0x7f729413d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f729413d900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f728800c780 tx=0x7f728800cc40 comp rx=0 tx=0).stop 2024-04-25T00:43:05.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 shutdown_connections 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f72780678f0 0x7f7278069db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f729413e710 0x7f72940aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72940cd410 0x7f72941f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 --2- 172.21.15.129:0/96240247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f729406b960 0x7f729413d900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 >> 172.21.15.129:0/96240247 conn(0x7f72940b2380 msgr2=0x7f72940cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 shutdown_connections 2024-04-25T00:43:05.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.308+0000 7f729ac36640 1 -- 172.21.15.129:0/96240247 wait complete. 2024-04-25T00:43:05.328 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T00:43:04.606607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.974660"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016159+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:43:00.016159+0000","last_peered":"2024-04-25T00:43:00.016159+0000","last_clean":"2024-04-25T00:43:00.016159+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:43:00.016159+0000","last_undegraded":"2024-04-25T00:43:00.016159+0000","last_fullsized":"2024-04-25T00:43:00.016159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657681+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:42:59.657681+0000","last_peered":"2024-04-25T00:42:59.657681+0000","last_clean":"2024-04-25T00:42:59.657681+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:42:59.657681+0000","last_undegraded":"2024-04-25T00:42:59.657681+0000","last_fullsized":"2024-04-25T00:42:59.657681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014295+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:43:00.014295+0000","last_peered":"2024-04-25T00:43:00.014295+0000","last_clean":"2024-04-25T00:43:00.014295+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:43:00.014295+0000","last_undegraded":"2024-04-25T00:43:00.014295+0000","last_fullsized":"2024-04-25T00:43:00.014295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013425+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:43:00.013425+0000","last_peered":"2024-04-25T00:43:00.013425+0000","last_clean":"2024-04-25T00:43:00.013425+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:43:00.013425+0000","last_undegraded":"2024-04-25T00:43:00.013425+0000","last_fullsized":"2024-04-25T00:43:00.013425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014816+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:43:00.014816+0000","last_peered":"2024-04-25T00:43:00.014816+0000","last_clean":"2024-04-25T00:43:00.014816+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:43:00.014816+0000","last_undegraded":"2024-04-25T00:43:00.014816+0000","last_fullsized":"2024-04-25T00:43:00.014816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013220+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:43:00.013220+0000","last_peered":"2024-04-25T00:43:00.013220+0000","last_clean":"2024-04-25T00:43:00.013220+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:43:00.013220+0000","last_undegraded":"2024-04-25T00:43:00.013220+0000","last_fullsized":"2024-04-25T00:43:00.013220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013356+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:43:00.013356+0000","last_peered":"2024-04-25T00:43:00.013356+0000","last_clean":"2024-04-25T00:43:00.013356+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:43:00.013356+0000","last_undegraded":"2024-04-25T00:43:00.013356+0000","last_fullsized":"2024-04-25T00:43:00.013356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940441+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:42:59.940441+0000","last_peered":"2024-04-25T00:42:59.940441+0000","last_clean":"2024-04-25T00:42:59.940441+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:42:59.940441+0000","last_undegraded":"2024-04-25T00:42:59.940441+0000","last_fullsized":"2024-04-25T00:42:59.940441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016200+0000","last_change":"2024-04-25T00:42:58.698697+0000","last_active":"2024-04-25T00:43:00.016200+0000","last_peered":"2024-04-25T00:43:00.016200+0000","last_clean":"2024-04-25T00:43:00.016200+0000","last_became_active":"2024-04-25T00:42:58.698442+0000","last_became_peered":"2024-04-25T00:42:58.698442+0000","last_unstale":"2024-04-25T00:43:00.016200+0000","last_undegraded":"2024-04-25T00:43:00.016200+0000","last_fullsized":"2024-04-25T00:43:00.016200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663628+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:42:59.663628+0000","last_peered":"2024-04-25T00:42:59.663628+0000","last_clean":"2024-04-25T00:42:59.663628+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:42:59.663628+0000","last_undegraded":"2024-04-25T00:42:59.663628+0000","last_fullsized":"2024-04-25T00:42:59.663628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663300+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:42:59.663300+0000","last_peered":"2024-04-25T00:42:59.663300+0000","last_clean":"2024-04-25T00:42:59.663300+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:42:59.663300+0000","last_undegraded":"2024-04-25T00:42:59.663300+0000","last_fullsized":"2024-04-25T00:42:59.663300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659638+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:42:59.659638+0000","last_peered":"2024-04-25T00:42:59.659638+0000","last_clean":"2024-04-25T00:42:59.659638+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:42:59.659638+0000","last_undegraded":"2024-04-25T00:42:59.659638+0000","last_fullsized":"2024-04-25T00:42:59.659638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940935+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:42:59.940935+0000","last_peered":"2024-04-25T00:42:59.940935+0000","last_clean":"2024-04-25T00:42:59.940935+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:42:59.940935+0000","last_undegraded":"2024-04-25T00:42:59.940935+0000","last_fullsized":"2024-04-25T00:42:59.940935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014187+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:43:00.014187+0000","last_peered":"2024-04-25T00:43:00.014187+0000","last_clean":"2024-04-25T00:43:00.014187+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:43:00.014187+0000","last_undegraded":"2024-04-25T00:43:00.014187+0000","last_fullsized":"2024-04-25T00:43:00.014187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656790+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:42:59.656790+0000","last_peered":"2024-04-25T00:42:59.656790+0000","last_clean":"2024-04-25T00:42:59.656790+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:42:59.656790+0000","last_undegraded":"2024-04-25T00:42:59.656790+0000","last_fullsized":"2024-04-25T00:42:59.656790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013062+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:43:00.013062+0000","last_peered":"2024-04-25T00:43:00.013062+0000","last_clean":"2024-04-25T00:43:00.013062+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:43:00.013062+0000","last_undegraded":"2024-04-25T00:43:00.013062+0000","last_fullsized":"2024-04-25T00:43:00.013062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014577+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:43:00.014577+0000","last_peered":"2024-04-25T00:43:00.014577+0000","last_clean":"2024-04-25T00:43:00.014577+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:43:00.014577+0000","last_undegraded":"2024-04-25T00:43:00.014577+0000","last_fullsized":"2024-04-25T00:43:00.014577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658598+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:42:59.658598+0000","last_peered":"2024-04-25T00:42:59.658598+0000","last_clean":"2024-04-25T00:42:59.658598+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:42:59.658598+0000","last_undegraded":"2024-04-25T00:42:59.658598+0000","last_fullsized":"2024-04-25T00:42:59.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015445+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:43:00.015445+0000","last_peered":"2024-04-25T00:43:00.015445+0000","last_clean":"2024-04-25T00:43:00.015445+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:43:00.015445+0000","last_undegraded":"2024-04-25T00:43:00.015445+0000","last_fullsized":"2024-04-25T00:43:00.015445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662931+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:42:59.662931+0000","last_peered":"2024-04-25T00:42:59.662931+0000","last_clean":"2024-04-25T00:42:59.662931+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:42:59.662931+0000","last_undegraded":"2024-04-25T00:42:59.662931+0000","last_fullsized":"2024-04-25T00:42:59.662931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014447+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:43:00.014447+0000","last_peered":"2024-04-25T00:43:00.014447+0000","last_clean":"2024-04-25T00:43:00.014447+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:43:00.014447+0000","last_undegraded":"2024-04-25T00:43:00.014447+0000","last_fullsized":"2024-04-25T00:43:00.014447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013170+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:43:00.013170+0000","last_peered":"2024-04-25T00:43:00.013170+0000","last_clean":"2024-04-25T00:43:00.013170+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:43:00.013170+0000","last_undegraded":"2024-04-25T00:43:00.013170+0000","last_fullsized":"2024-04-25T00:43:00.013170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940601+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:42:59.940601+0000","last_peered":"2024-04-25T00:42:59.940601+0000","last_clean":"2024-04-25T00:42:59.940601+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:42:59.940601+0000","last_undegraded":"2024-04-25T00:42:59.940601+0000","last_fullsized":"2024-04-25T00:42:59.940601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012818+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:43:00.012818+0000","last_peered":"2024-04-25T00:43:00.012818+0000","last_clean":"2024-04-25T00:43:00.012818+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:43:00.012818+0000","last_undegraded":"2024-04-25T00:43:00.012818+0000","last_fullsized":"2024-04-25T00:43:00.012818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941753+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:42:59.941753+0000","last_peered":"2024-04-25T00:42:59.941753+0000","last_clean":"2024-04-25T00:42:59.941753+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:42:59.941753+0000","last_undegraded":"2024-04-25T00:42:59.941753+0000","last_fullsized":"2024-04-25T00:42:59.941753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663637+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:42:59.663637+0000","last_peered":"2024-04-25T00:42:59.663637+0000","last_clean":"2024-04-25T00:42:59.663637+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:42:59.663637+0000","last_undegraded":"2024-04-25T00:42:59.663637+0000","last_fullsized":"2024-04-25T00:42:59.663637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657292+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:42:59.657292+0000","last_peered":"2024-04-25T00:42:59.657292+0000","last_clean":"2024-04-25T00:42:59.657292+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:42:59.657292+0000","last_undegraded":"2024-04-25T00:42:59.657292+0000","last_fullsized":"2024-04-25T00:42:59.657292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941489+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:42:59.941489+0000","last_peered":"2024-04-25T00:42:59.941489+0000","last_clean":"2024-04-25T00:42:59.941489+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:42:59.941489+0000","last_undegraded":"2024-04-25T00:42:59.941489+0000","last_fullsized":"2024-04-25T00:42:59.941489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014644+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:43:00.014644+0000","last_peered":"2024-04-25T00:43:00.014644+0000","last_clean":"2024-04-25T00:43:00.014644+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:43:00.014644+0000","last_undegraded":"2024-04-25T00:43:00.014644+0000","last_fullsized":"2024-04-25T00:43:00.014644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013656+0000","last_change":"2024-04-25T00:42:58.693414+0000","last_active":"2024-04-25T00:43:00.013656+0000","last_peered":"2024-04-25T00:43:00.013656+0000","last_clean":"2024-04-25T00:43:00.013656+0000","last_became_active":"2024-04-25T00:42:58.693071+0000","last_became_peered":"2024-04-25T00:42:58.693071+0000","last_unstale":"2024-04-25T00:43:00.013656+0000","last_undegraded":"2024-04-25T00:43:00.013656+0000","last_fullsized":"2024-04-25T00:43:00.013656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658896+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:42:59.658896+0000","last_peered":"2024-04-25T00:42:59.658896+0000","last_clean":"2024-04-25T00:42:59.658896+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:42:59.658896+0000","last_undegraded":"2024-04-25T00:42:59.658896+0000","last_fullsized":"2024-04-25T00:42:59.658896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662637+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:42:59.662637+0000","last_peered":"2024-04-25T00:42:59.662637+0000","last_clean":"2024-04-25T00:42:59.662637+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:42:59.662637+0000","last_undegraded":"2024-04-25T00:42:59.662637+0000","last_fullsized":"2024-04-25T00:42:59.662637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658748+0000","last_change":"2024-04-25T00:42:58.689399+0000","last_active":"2024-04-25T00:42:59.658748+0000","last_peered":"2024-04-25T00:42:59.658748+0000","last_clean":"2024-04-25T00:42:59.658748+0000","last_became_active":"2024-04-25T00:42:58.689167+0000","last_became_peered":"2024-04-25T00:42:58.689167+0000","last_unstale":"2024-04-25T00:42:59.658748+0000","last_undegraded":"2024-04-25T00:42:59.658748+0000","last_fullsized":"2024-04-25T00:42:59.658748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:06.431173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941031+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:42:59.941031+0000","last_peered":"2024-04-25T00:42:59.941031+0000","last_clean":"2024-04-25T00:42:59.941031+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:42:59.941031+0000","last_undegraded":"2024-04-25T00:42:59.941031+0000","last_fullsized":"2024-04-25T00:42:59.941031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014851+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:43:00.014851+0000","last_peered":"2024-04-25T00:43:00.014851+0000","last_clean":"2024-04-25T00:43:00.014851+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:43:00.014851+0000","last_undegraded":"2024-04-25T00:43:00.014851+0000","last_fullsized":"2024-04-25T00:43:00.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662544+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:42:59.662544+0000","last_peered":"2024-04-25T00:42:59.662544+0000","last_clean":"2024-04-25T00:42:59.662544+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:42:59.662544+0000","last_undegraded":"2024-04-25T00:42:59.662544+0000","last_fullsized":"2024-04-25T00:42:59.662544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940666+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:42:59.940666+0000","last_peered":"2024-04-25T00:42:59.940666+0000","last_clean":"2024-04-25T00:42:59.940666+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:42:59.940666+0000","last_undegraded":"2024-04-25T00:42:59.940666+0000","last_fullsized":"2024-04-25T00:42:59.940666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940621+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:42:59.940621+0000","last_peered":"2024-04-25T00:42:59.940621+0000","last_clean":"2024-04-25T00:42:59.940621+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:42:59.940621+0000","last_undegraded":"2024-04-25T00:42:59.940621+0000","last_fullsized":"2024-04-25T00:42:59.940621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012893+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:43:00.012893+0000","last_peered":"2024-04-25T00:43:00.012893+0000","last_clean":"2024-04-25T00:43:00.012893+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:43:00.012893+0000","last_undegraded":"2024-04-25T00:43:00.012893+0000","last_fullsized":"2024-04-25T00:43:00.012893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656623+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:42:59.656623+0000","last_peered":"2024-04-25T00:42:59.656623+0000","last_clean":"2024-04-25T00:42:59.656623+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:42:59.656623+0000","last_undegraded":"2024-04-25T00:42:59.656623+0000","last_fullsized":"2024-04-25T00:42:59.656623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663244+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:42:59.663244+0000","last_peered":"2024-04-25T00:42:59.663244+0000","last_clean":"2024-04-25T00:42:59.663244+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:42:59.663244+0000","last_undegraded":"2024-04-25T00:42:59.663244+0000","last_fullsized":"2024-04-25T00:42:59.663244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013906+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:43:00.013906+0000","last_peered":"2024-04-25T00:43:00.013906+0000","last_clean":"2024-04-25T00:43:00.013906+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:43:00.013906+0000","last_undegraded":"2024-04-25T00:43:00.013906+0000","last_fullsized":"2024-04-25T00:43:00.013906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013989+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:43:00.013989+0000","last_peered":"2024-04-25T00:43:00.013989+0000","last_clean":"2024-04-25T00:43:00.013989+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:43:00.013989+0000","last_undegraded":"2024-04-25T00:43:00.013989+0000","last_fullsized":"2024-04-25T00:43:00.013989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014621+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:43:00.014621+0000","last_peered":"2024-04-25T00:43:00.014621+0000","last_clean":"2024-04-25T00:43:00.014621+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:43:00.014621+0000","last_undegraded":"2024-04-25T00:43:00.014621+0000","last_fullsized":"2024-04-25T00:43:00.014621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657481+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:42:59.657481+0000","last_peered":"2024-04-25T00:42:59.657481+0000","last_clean":"2024-04-25T00:42:59.657481+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:42:59.657481+0000","last_undegraded":"2024-04-25T00:42:59.657481+0000","last_fullsized":"2024-04-25T00:42:59.657481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659786+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:42:59.659786+0000","last_peered":"2024-04-25T00:42:59.659786+0000","last_clean":"2024-04-25T00:42:59.659786+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:42:59.659786+0000","last_undegraded":"2024-04-25T00:42:59.659786+0000","last_fullsized":"2024-04-25T00:42:59.659786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662829+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:42:59.662829+0000","last_peered":"2024-04-25T00:42:59.662829+0000","last_clean":"2024-04-25T00:42:59.662829+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:42:59.662829+0000","last_undegraded":"2024-04-25T00:42:59.662829+0000","last_fullsized":"2024-04-25T00:42:59.662829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015191+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:43:00.015191+0000","last_peered":"2024-04-25T00:43:00.015191+0000","last_clean":"2024-04-25T00:43:00.015191+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:43:00.015191+0000","last_undegraded":"2024-04-25T00:43:00.015191+0000","last_fullsized":"2024-04-25T00:43:00.015191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014789+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:43:00.014789+0000","last_peered":"2024-04-25T00:43:00.014789+0000","last_clean":"2024-04-25T00:43:00.014789+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:43:00.014789+0000","last_undegraded":"2024-04-25T00:43:00.014789+0000","last_fullsized":"2024-04-25T00:43:00.014789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014462+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:43:00.014462+0000","last_peered":"2024-04-25T00:43:00.014462+0000","last_clean":"2024-04-25T00:43:00.014462+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:43:00.014462+0000","last_undegraded":"2024-04-25T00:43:00.014462+0000","last_fullsized":"2024-04-25T00:43:00.014462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013395+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:43:00.013395+0000","last_peered":"2024-04-25T00:43:00.013395+0000","last_clean":"2024-04-25T00:43:00.013395+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:43:00.013395+0000","last_undegraded":"2024-04-25T00:43:00.013395+0000","last_fullsized":"2024-04-25T00:43:00.013395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014992+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:43:00.014992+0000","last_peered":"2024-04-25T00:43:00.014992+0000","last_clean":"2024-04-25T00:43:00.014992+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:43:00.014992+0000","last_undegraded":"2024-04-25T00:43:00.014992+0000","last_fullsized":"2024-04-25T00:43:00.014992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662429+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:42:59.662429+0000","last_peered":"2024-04-25T00:42:59.662429+0000","last_clean":"2024-04-25T00:42:59.662429+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:42:59.662429+0000","last_undegraded":"2024-04-25T00:42:59.662429+0000","last_fullsized":"2024-04-25T00:42:59.662429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662799+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:42:59.662799+0000","last_peered":"2024-04-25T00:42:59.662799+0000","last_clean":"2024-04-25T00:42:59.662799+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:42:59.662799+0000","last_undegraded":"2024-04-25T00:42:59.662799+0000","last_fullsized":"2024-04-25T00:42:59.662799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658133+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:42:59.658133+0000","last_peered":"2024-04-25T00:42:59.658133+0000","last_clean":"2024-04-25T00:42:59.658133+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:42:59.658133+0000","last_undegraded":"2024-04-25T00:42:59.658133+0000","last_fullsized":"2024-04-25T00:42:59.658133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014619+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:43:00.014619+0000","last_peered":"2024-04-25T00:43:00.014619+0000","last_clean":"2024-04-25T00:43:00.014619+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:43:00.014619+0000","last_undegraded":"2024-04-25T00:43:00.014619+0000","last_fullsized":"2024-04-25T00:43:00.014619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659249+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:42:59.659249+0000","last_peered":"2024-04-25T00:42:59.659249+0000","last_clean":"2024-04-25T00:42:59.659249+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:42:59.659249+0000","last_undegraded":"2024-04-25T00:42:59.659249+0000","last_fullsized":"2024-04-25T00:42:59.659249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015947+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:43:00.015947+0000","last_peered":"2024-04-25T00:43:00.015947+0000","last_clean":"2024-04-25T00:43:00.015947+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:43:00.015947+0000","last_undegraded":"2024-04-25T00:43:00.015947+0000","last_fullsized":"2024-04-25T00:43:00.015947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658685+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:42:59.658685+0000","last_peered":"2024-04-25T00:42:59.658685+0000","last_clean":"2024-04-25T00:42:59.658685+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:42:59.658685+0000","last_undegraded":"2024-04-25T00:42:59.658685+0000","last_fullsized":"2024-04-25T00:42:59.658685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016444+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:43:00.016444+0000","last_peered":"2024-04-25T00:43:00.016444+0000","last_clean":"2024-04-25T00:43:00.016444+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:43:00.016444+0000","last_undegraded":"2024-04-25T00:43:00.016444+0000","last_fullsized":"2024-04-25T00:43:00.016444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657468+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:42:59.657468+0000","last_peered":"2024-04-25T00:42:59.657468+0000","last_clean":"2024-04-25T00:42:59.657468+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:42:59.657468+0000","last_undegraded":"2024-04-25T00:42:59.657468+0000","last_fullsized":"2024-04-25T00:42:59.657468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013590+0000","last_change":"2024-04-25T00:42:58.693586+0000","last_active":"2024-04-25T00:43:00.013590+0000","last_peered":"2024-04-25T00:43:00.013590+0000","last_clean":"2024-04-25T00:43:00.013590+0000","last_became_active":"2024-04-25T00:42:58.692506+0000","last_became_peered":"2024-04-25T00:42:58.692506+0000","last_unstale":"2024-04-25T00:43:00.013590+0000","last_undegraded":"2024-04-25T00:43:00.013590+0000","last_fullsized":"2024-04-25T00:43:00.013590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658281+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:42:59.658281+0000","last_peered":"2024-04-25T00:42:59.658281+0000","last_clean":"2024-04-25T00:42:59.658281+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:42:59.658281+0000","last_undegraded":"2024-04-25T00:42:59.658281+0000","last_fullsized":"2024-04-25T00:42:59.658281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015234+0000","last_change":"2024-04-25T00:42:58.698024+0000","last_active":"2024-04-25T00:43:00.015234+0000","last_peered":"2024-04-25T00:43:00.015234+0000","last_clean":"2024-04-25T00:43:00.015234+0000","last_became_active":"2024-04-25T00:42:58.697733+0000","last_became_peered":"2024-04-25T00:42:58.697733+0000","last_unstale":"2024-04-25T00:43:00.015234+0000","last_undegraded":"2024-04-25T00:43:00.015234+0000","last_fullsized":"2024-04-25T00:43:00.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013464+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:00.013464+0000","last_peered":"2024-04-25T00:43:00.013464+0000","last_clean":"2024-04-25T00:43:00.013464+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:00.013464+0000","last_undegraded":"2024-04-25T00:43:00.013464+0000","last_fullsized":"2024-04-25T00:43:00.013464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013023+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:43:00.013023+0000","last_peered":"2024-04-25T00:43:00.013023+0000","last_clean":"2024-04-25T00:43:00.013023+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:43:00.013023+0000","last_undegraded":"2024-04-25T00:43:00.013023+0000","last_fullsized":"2024-04-25T00:43:00.013023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941039+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:42:59.941039+0000","last_peered":"2024-04-25T00:42:59.941039+0000","last_clean":"2024-04-25T00:42:59.941039+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:42:59.941039+0000","last_undegraded":"2024-04-25T00:42:59.941039+0000","last_fullsized":"2024-04-25T00:42:59.941039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013385+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:43:00.013385+0000","last_peered":"2024-04-25T00:43:00.013385+0000","last_clean":"2024-04-25T00:43:00.013385+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:43:00.013385+0000","last_undegraded":"2024-04-25T00:43:00.013385+0000","last_fullsized":"2024-04-25T00:43:00.013385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015080+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:43:00.015080+0000","last_peered":"2024-04-25T00:43:00.015080+0000","last_clean":"2024-04-25T00:43:00.015080+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:43:00.015080+0000","last_undegraded":"2024-04-25T00:43:00.015080+0000","last_fullsized":"2024-04-25T00:43:00.015080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658309+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:42:59.658309+0000","last_peered":"2024-04-25T00:42:59.658309+0000","last_clean":"2024-04-25T00:42:59.658309+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:42:59.658309+0000","last_undegraded":"2024-04-25T00:42:59.658309+0000","last_fullsized":"2024-04-25T00:42:59.658309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656914+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:42:59.656914+0000","last_peered":"2024-04-25T00:42:59.656914+0000","last_clean":"2024-04-25T00:42:59.656914+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:42:59.656914+0000","last_undegraded":"2024-04-25T00:42:59.656914+0000","last_fullsized":"2024-04-25T00:42:59.656914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013694+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:43:00.013694+0000","last_peered":"2024-04-25T00:43:00.013694+0000","last_clean":"2024-04-25T00:43:00.013694+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:43:00.013694+0000","last_undegraded":"2024-04-25T00:43:00.013694+0000","last_fullsized":"2024-04-25T00:43:00.013694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659040+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:42:59.659040+0000","last_peered":"2024-04-25T00:42:59.659040+0000","last_clean":"2024-04-25T00:42:59.659040+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:42:59.659040+0000","last_undegraded":"2024-04-25T00:42:59.659040+0000","last_fullsized":"2024-04-25T00:42:59.659040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657076+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:42:59.657076+0000","last_peered":"2024-04-25T00:42:59.657076+0000","last_clean":"2024-04-25T00:42:59.657076+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:42:59.657076+0000","last_undegraded":"2024-04-25T00:42:59.657076+0000","last_fullsized":"2024-04-25T00:42:59.657076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014402+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:43:00.014402+0000","last_peered":"2024-04-25T00:43:00.014402+0000","last_clean":"2024-04-25T00:43:00.014402+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:43:00.014402+0000","last_undegraded":"2024-04-25T00:43:00.014402+0000","last_fullsized":"2024-04-25T00:43:00.014402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015510+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:43:00.015510+0000","last_peered":"2024-04-25T00:43:00.015510+0000","last_clean":"2024-04-25T00:43:00.015510+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:43:00.015510+0000","last_undegraded":"2024-04-25T00:43:00.015510+0000","last_fullsized":"2024-04-25T00:43:00.015510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656937+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:42:59.656937+0000","last_peered":"2024-04-25T00:42:59.656937+0000","last_clean":"2024-04-25T00:42:59.656937+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:42:59.656937+0000","last_undegraded":"2024-04-25T00:42:59.656937+0000","last_fullsized":"2024-04-25T00:42:59.656937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657625+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:42:59.657625+0000","last_peered":"2024-04-25T00:42:59.657625+0000","last_clean":"2024-04-25T00:42:59.657625+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:42:59.657625+0000","last_undegraded":"2024-04-25T00:42:59.657625+0000","last_fullsized":"2024-04-25T00:42:59.657625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941751+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:42:59.941751+0000","last_peered":"2024-04-25T00:42:59.941751+0000","last_clean":"2024-04-25T00:42:59.941751+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:42:59.941751+0000","last_undegraded":"2024-04-25T00:42:59.941751+0000","last_fullsized":"2024-04-25T00:42:59.941751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663349+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:42:59.663349+0000","last_peered":"2024-04-25T00:42:59.663349+0000","last_clean":"2024-04-25T00:42:59.663349+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:42:59.663349+0000","last_undegraded":"2024-04-25T00:42:59.663349+0000","last_fullsized":"2024-04-25T00:42:59.663349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014387+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:43:00.014387+0000","last_peered":"2024-04-25T00:43:00.014387+0000","last_clean":"2024-04-25T00:43:00.014387+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:43:00.014387+0000","last_undegraded":"2024-04-25T00:43:00.014387+0000","last_fullsized":"2024-04-25T00:43:00.014387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013174+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:43:00.013174+0000","last_peered":"2024-04-25T00:43:00.013174+0000","last_clean":"2024-04-25T00:43:00.013174+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:43:00.013174+0000","last_undegraded":"2024-04-25T00:43:00.013174+0000","last_fullsized":"2024-04-25T00:43:00.013174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659268+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:42:59.659268+0000","last_peered":"2024-04-25T00:42:59.659268+0000","last_clean":"2024-04-25T00:42:59.659268+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:42:59.659268+0000","last_undegraded":"2024-04-25T00:42:59.659268+0000","last_fullsized":"2024-04-25T00:42:59.659268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656849+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:42:59.656849+0000","last_peered":"2024-04-25T00:42:59.656849+0000","last_clean":"2024-04-25T00:42:59.656849+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:42:59.656849+0000","last_undegraded":"2024-04-25T00:42:59.656849+0000","last_fullsized":"2024-04-25T00:42:59.656849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941104+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:42:59.941104+0000","last_peered":"2024-04-25T00:42:59.941104+0000","last_clean":"2024-04-25T00:42:59.941104+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:42:59.941104+0000","last_undegraded":"2024-04-25T00:42:59.941104+0000","last_fullsized":"2024-04-25T00:42:59.941104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657161+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:42:59.657161+0000","last_peered":"2024-04-25T00:42:59.657161+0000","last_clean":"2024-04-25T00:42:59.657161+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:42:59.657161+0000","last_undegraded":"2024-04-25T00:42:59.657161+0000","last_fullsized":"2024-04-25T00:42:59.657161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941098+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:42:59.941098+0000","last_peered":"2024-04-25T00:42:59.941098+0000","last_clean":"2024-04-25T00:42:59.941098+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:42:59.941098+0000","last_undegraded":"2024-04-25T00:42:59.941098+0000","last_fullsized":"2024-04-25T00:42:59.941098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015070+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:43:00.015070+0000","last_peered":"2024-04-25T00:43:00.015070+0000","last_clean":"2024-04-25T00:43:00.015070+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:43:00.015070+0000","last_undegraded":"2024-04-25T00:43:00.015070+0000","last_fullsized":"2024-04-25T00:43:00.015070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662834+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:42:59.662834+0000","last_peered":"2024-04-25T00:42:59.662834+0000","last_clean":"2024-04-25T00:42:59.662834+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:42:59.662834+0000","last_undegraded":"2024-04-25T00:42:59.662834+0000","last_fullsized":"2024-04-25T00:42:59.662834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014334+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:43:00.014334+0000","last_peered":"2024-04-25T00:43:00.014334+0000","last_clean":"2024-04-25T00:43:00.014334+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:43:00.014334+0000","last_undegraded":"2024-04-25T00:43:00.014334+0000","last_fullsized":"2024-04-25T00:43:00.014334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014890+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:43:00.014890+0000","last_peered":"2024-04-25T00:43:00.014890+0000","last_clean":"2024-04-25T00:43:00.014890+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:43:00.014890+0000","last_undegraded":"2024-04-25T00:43:00.014890+0000","last_fullsized":"2024-04-25T00:43:00.014890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013948+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:43:00.013948+0000","last_peered":"2024-04-25T00:43:00.013948+0000","last_clean":"2024-04-25T00:43:00.013948+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:43:00.013948+0000","last_undegraded":"2024-04-25T00:43:00.013948+0000","last_fullsized":"2024-04-25T00:43:00.013948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015334+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:43:00.015334+0000","last_peered":"2024-04-25T00:43:00.015334+0000","last_clean":"2024-04-25T00:43:00.015334+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:43:00.015334+0000","last_undegraded":"2024-04-25T00:43:00.015334+0000","last_fullsized":"2024-04-25T00:43:00.015334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014217+0000","last_change":"2024-04-25T00:42:58.700939+0000","last_active":"2024-04-25T00:43:00.014217+0000","last_peered":"2024-04-25T00:43:00.014217+0000","last_clean":"2024-04-25T00:43:00.014217+0000","last_became_active":"2024-04-25T00:42:58.700177+0000","last_became_peered":"2024-04-25T00:42:58.700177+0000","last_unstale":"2024-04-25T00:43:00.014217+0000","last_undegraded":"2024-04-25T00:43:00.014217+0000","last_fullsized":"2024-04-25T00:43:00.014217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:36.086387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941542+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:42:59.941542+0000","last_peered":"2024-04-25T00:42:59.941542+0000","last_clean":"2024-04-25T00:42:59.941542+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:42:59.941542+0000","last_undegraded":"2024-04-25T00:42:59.941542+0000","last_fullsized":"2024-04-25T00:42:59.941542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941612+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:42:59.941612+0000","last_peered":"2024-04-25T00:42:59.941612+0000","last_clean":"2024-04-25T00:42:59.941612+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:42:59.941612+0000","last_undegraded":"2024-04-25T00:42:59.941612+0000","last_fullsized":"2024-04-25T00:42:59.941612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015091+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:43:00.015091+0000","last_peered":"2024-04-25T00:43:00.015091+0000","last_clean":"2024-04-25T00:43:00.015091+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:43:00.015091+0000","last_undegraded":"2024-04-25T00:43:00.015091+0000","last_fullsized":"2024-04-25T00:43:00.015091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:43:05.332 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:43:05.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e1d5bc640 1 Processor -- start 2024-04-25T00:43:05.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e1d5bc640 1 -- start start 2024-04-25T00:43:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e1d5bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180ccdb0 0x7f4e180723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e1d5bc640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e18072aa0 con 0x7f4e180cd190 2024-04-25T00:43:05.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e1d5bc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e180708e0 con 0x7f4e1815c7f0 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e1d5bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4e18070690 con 0x7f4e180ccdb0 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f4e180cd190 0x7f4e1806de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46734/0 (socket says 172.21.15.129:46734) 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 -- 172.21.15.129:0/3858771825 learned_addr learned my addr 172.21.15.129:0/3858771825 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e16ffd640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180ccdb0 0x7f4e180723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112272858 0 0) 0x7f4e18072aa0 con 0x7f4e180cd190 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e00003530 con 0x7f4e180cd190 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 819712814 0 0) 0x7f4e00003530 con 0x7f4e180cd190 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180ccdb0 msgr2=0x7f4e180723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180ccdb0 0x7f4e180723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 >> v1:172.21.15.196:6789/0 conn(0x7f4e1815c7f0 legacy=0x7f4e18158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e18070bc0 con 0x7f4e180cd190 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f4e10001680 con 0x7f4e180cd190 2024-04-25T00:43:05.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 >> v1:172.21.15.129:6789/0 conn(0x7f4e180cd190 legacy=0x7f4e1806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e000039f0 0x7f4e00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e00004810 0x7f4e00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18070690 con 0x7f4e000052f0 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0000c170 con 0x7f4e00004810 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0000c370 con 0x7f4e000039f0 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e16ffd640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180ccdb0 0x7f4e180723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e167fc640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e00004810 0x7f4e00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 -- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e000039f0 msgr2=0x7f4e00003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e000039f0 0x7f4e00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 -- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e00004810 msgr2=0x7f4e00004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e00004810 0x7f4e00004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.480+0000 7f4e177fe640 1 -- 172.21.15.129:0/3858771825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e00003530 con 0x7f4e000052f0 2024-04-25T00:43:05.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e177fe640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e0000b9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c110 tx=0x7f4e1001f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1c218d514f32f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10007c30 con 0x7f4e000052f0 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e10005b00 con 0x7f4e000052f0 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e15ffb640 1 -- 172.21.15.129:0/3858771825 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10007070 con 0x7f4e000052f0 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/3858771825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 msgr2=0x7f4e0000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/3858771825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e0000b9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c110 tx=0x7f4e1001f2f0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e177fe640 1 -- 172.21.15.129:0/3858771825 reap_dead start 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/3858771825 shutdown_connections 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/3858771825 >> 172.21.15.129:0/3858771825 conn(0x7f4e180b14a0 msgr2=0x7f4e180b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/3858771825 shutdown_connections 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/3858771825 wait complete. 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 Processor -- start 2024-04-25T00:43:05.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- start start 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e1815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e180c5e30 0x7f4e181551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180c4580 0x7f4e180ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e180723e0 con 0x7f4e000052f0 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18158fa0 con 0x7f4e180c5e30 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e1d5bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4e18072560 con 0x7f4e180c4580 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e1815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e1815a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42898/0 (socket says 172.21.15.129:42898) 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.484+0000 7f4e16ffd640 1 -- 172.21.15.129:0/2466867917 learned_addr learned my addr 172.21.15.129:0/2466867917 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e177fe640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180c4580 0x7f4e180ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e16ffd640 1 -- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180c4580 msgr2=0x7f4e180ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e16ffd640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180c4580 0x7f4e180ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e16ffd640 1 -- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e180c5e30 msgr2=0x7f4e181551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e16ffd640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e180c5e30 0x7f4e181551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e16ffd640 1 -- 172.21.15.129:0/2466867917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e18155700 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e16ffd640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e1815a970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800d410 tx=0x7f4e0800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08004020 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e080041e0 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e180c4d90 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080043d0 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e180c5350 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e18062e50 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e0801a020 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4dff7fe640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4e040678f0 0x7f4e04069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e08085670 con 0x7f4e000052f0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e167fc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4e040678f0 0x7f4e04069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.488+0000 7f4e167fc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4e040678f0 0x7f4e04069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4e180b11b0 tx=0x7f4e0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.492+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4e080561b0 con 0x7f4e000052f0 2024-04-25T00:43:05.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.636+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e18075320 con 0x7f4e040678f0 2024-04-25T00:43:05.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.656+0000 7f4dff7fe640 1 -- 172.21.15.129:0/2466867917 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f4e18075320 con 0x7f4e040678f0 2024-04-25T00:43:05.660 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:05.662 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:43:05.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4e040678f0 msgr2=0x7f4e04069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4e040678f0 0x7f4e04069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4e180b11b0 tx=0x7f4e0c009040 comp rx=0 tx=0).stop 2024-04-25T00:43:05.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 msgr2=0x7f4e1815a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e1815a970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800d410 tx=0x7f4e0800d8d0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 shutdown_connections 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4e040678f0 0x7f4e04069db0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4e180c4580 0x7f4e180ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e180c5e30 0x7f4e181551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 --2- 172.21.15.129:0/2466867917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e000052f0 0x7f4e1815a970 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 >> 172.21.15.129:0/2466867917 conn(0x7f4e180b14a0 msgr2=0x7f4e1815d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 shutdown_connections 2024-04-25T00:43:05.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.660+0000 7f4e1d5bc640 1 -- 172.21.15.129:0/2466867917 wait complete. 2024-04-25T00:43:05.678 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T00:43:04.606607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.974660"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016159+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:43:00.016159+0000","last_peered":"2024-04-25T00:43:00.016159+0000","last_clean":"2024-04-25T00:43:00.016159+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:43:00.016159+0000","last_undegraded":"2024-04-25T00:43:00.016159+0000","last_fullsized":"2024-04-25T00:43:00.016159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657681+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:42:59.657681+0000","last_peered":"2024-04-25T00:42:59.657681+0000","last_clean":"2024-04-25T00:42:59.657681+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:42:59.657681+0000","last_undegraded":"2024-04-25T00:42:59.657681+0000","last_fullsized":"2024-04-25T00:42:59.657681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014295+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:43:00.014295+0000","last_peered":"2024-04-25T00:43:00.014295+0000","last_clean":"2024-04-25T00:43:00.014295+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:43:00.014295+0000","last_undegraded":"2024-04-25T00:43:00.014295+0000","last_fullsized":"2024-04-25T00:43:00.014295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013425+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:43:00.013425+0000","last_peered":"2024-04-25T00:43:00.013425+0000","last_clean":"2024-04-25T00:43:00.013425+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:43:00.013425+0000","last_undegraded":"2024-04-25T00:43:00.013425+0000","last_fullsized":"2024-04-25T00:43:00.013425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014816+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:43:00.014816+0000","last_peered":"2024-04-25T00:43:00.014816+0000","last_clean":"2024-04-25T00:43:00.014816+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:43:00.014816+0000","last_undegraded":"2024-04-25T00:43:00.014816+0000","last_fullsized":"2024-04-25T00:43:00.014816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013220+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:43:00.013220+0000","last_peered":"2024-04-25T00:43:00.013220+0000","last_clean":"2024-04-25T00:43:00.013220+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:43:00.013220+0000","last_undegraded":"2024-04-25T00:43:00.013220+0000","last_fullsized":"2024-04-25T00:43:00.013220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013356+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:43:00.013356+0000","last_peered":"2024-04-25T00:43:00.013356+0000","last_clean":"2024-04-25T00:43:00.013356+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:43:00.013356+0000","last_undegraded":"2024-04-25T00:43:00.013356+0000","last_fullsized":"2024-04-25T00:43:00.013356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940441+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:42:59.940441+0000","last_peered":"2024-04-25T00:42:59.940441+0000","last_clean":"2024-04-25T00:42:59.940441+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:42:59.940441+0000","last_undegraded":"2024-04-25T00:42:59.940441+0000","last_fullsized":"2024-04-25T00:42:59.940441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016200+0000","last_change":"2024-04-25T00:42:58.698697+0000","last_active":"2024-04-25T00:43:00.016200+0000","last_peered":"2024-04-25T00:43:00.016200+0000","last_clean":"2024-04-25T00:43:00.016200+0000","last_became_active":"2024-04-25T00:42:58.698442+0000","last_became_peered":"2024-04-25T00:42:58.698442+0000","last_unstale":"2024-04-25T00:43:00.016200+0000","last_undegraded":"2024-04-25T00:43:00.016200+0000","last_fullsized":"2024-04-25T00:43:00.016200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663628+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:42:59.663628+0000","last_peered":"2024-04-25T00:42:59.663628+0000","last_clean":"2024-04-25T00:42:59.663628+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:42:59.663628+0000","last_undegraded":"2024-04-25T00:42:59.663628+0000","last_fullsized":"2024-04-25T00:42:59.663628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663300+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:42:59.663300+0000","last_peered":"2024-04-25T00:42:59.663300+0000","last_clean":"2024-04-25T00:42:59.663300+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:42:59.663300+0000","last_undegraded":"2024-04-25T00:42:59.663300+0000","last_fullsized":"2024-04-25T00:42:59.663300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659638+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:42:59.659638+0000","last_peered":"2024-04-25T00:42:59.659638+0000","last_clean":"2024-04-25T00:42:59.659638+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:42:59.659638+0000","last_undegraded":"2024-04-25T00:42:59.659638+0000","last_fullsized":"2024-04-25T00:42:59.659638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940935+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:42:59.940935+0000","last_peered":"2024-04-25T00:42:59.940935+0000","last_clean":"2024-04-25T00:42:59.940935+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:42:59.940935+0000","last_undegraded":"2024-04-25T00:42:59.940935+0000","last_fullsized":"2024-04-25T00:42:59.940935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014187+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:43:00.014187+0000","last_peered":"2024-04-25T00:43:00.014187+0000","last_clean":"2024-04-25T00:43:00.014187+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:43:00.014187+0000","last_undegraded":"2024-04-25T00:43:00.014187+0000","last_fullsized":"2024-04-25T00:43:00.014187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656790+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:42:59.656790+0000","last_peered":"2024-04-25T00:42:59.656790+0000","last_clean":"2024-04-25T00:42:59.656790+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:42:59.656790+0000","last_undegraded":"2024-04-25T00:42:59.656790+0000","last_fullsized":"2024-04-25T00:42:59.656790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013062+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:43:00.013062+0000","last_peered":"2024-04-25T00:43:00.013062+0000","last_clean":"2024-04-25T00:43:00.013062+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:43:00.013062+0000","last_undegraded":"2024-04-25T00:43:00.013062+0000","last_fullsized":"2024-04-25T00:43:00.013062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014577+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:43:00.014577+0000","last_peered":"2024-04-25T00:43:00.014577+0000","last_clean":"2024-04-25T00:43:00.014577+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:43:00.014577+0000","last_undegraded":"2024-04-25T00:43:00.014577+0000","last_fullsized":"2024-04-25T00:43:00.014577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658598+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:42:59.658598+0000","last_peered":"2024-04-25T00:42:59.658598+0000","last_clean":"2024-04-25T00:42:59.658598+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:42:59.658598+0000","last_undegraded":"2024-04-25T00:42:59.658598+0000","last_fullsized":"2024-04-25T00:42:59.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015445+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:43:00.015445+0000","last_peered":"2024-04-25T00:43:00.015445+0000","last_clean":"2024-04-25T00:43:00.015445+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:43:00.015445+0000","last_undegraded":"2024-04-25T00:43:00.015445+0000","last_fullsized":"2024-04-25T00:43:00.015445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662931+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:42:59.662931+0000","last_peered":"2024-04-25T00:42:59.662931+0000","last_clean":"2024-04-25T00:42:59.662931+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:42:59.662931+0000","last_undegraded":"2024-04-25T00:42:59.662931+0000","last_fullsized":"2024-04-25T00:42:59.662931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014447+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:43:00.014447+0000","last_peered":"2024-04-25T00:43:00.014447+0000","last_clean":"2024-04-25T00:43:00.014447+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:43:00.014447+0000","last_undegraded":"2024-04-25T00:43:00.014447+0000","last_fullsized":"2024-04-25T00:43:00.014447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013170+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:43:00.013170+0000","last_peered":"2024-04-25T00:43:00.013170+0000","last_clean":"2024-04-25T00:43:00.013170+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:43:00.013170+0000","last_undegraded":"2024-04-25T00:43:00.013170+0000","last_fullsized":"2024-04-25T00:43:00.013170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940601+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:42:59.940601+0000","last_peered":"2024-04-25T00:42:59.940601+0000","last_clean":"2024-04-25T00:42:59.940601+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:42:59.940601+0000","last_undegraded":"2024-04-25T00:42:59.940601+0000","last_fullsized":"2024-04-25T00:42:59.940601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012818+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:43:00.012818+0000","last_peered":"2024-04-25T00:43:00.012818+0000","last_clean":"2024-04-25T00:43:00.012818+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:43:00.012818+0000","last_undegraded":"2024-04-25T00:43:00.012818+0000","last_fullsized":"2024-04-25T00:43:00.012818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941753+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:42:59.941753+0000","last_peered":"2024-04-25T00:42:59.941753+0000","last_clean":"2024-04-25T00:42:59.941753+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:42:59.941753+0000","last_undegraded":"2024-04-25T00:42:59.941753+0000","last_fullsized":"2024-04-25T00:42:59.941753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663637+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:42:59.663637+0000","last_peered":"2024-04-25T00:42:59.663637+0000","last_clean":"2024-04-25T00:42:59.663637+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:42:59.663637+0000","last_undegraded":"2024-04-25T00:42:59.663637+0000","last_fullsized":"2024-04-25T00:42:59.663637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657292+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:42:59.657292+0000","last_peered":"2024-04-25T00:42:59.657292+0000","last_clean":"2024-04-25T00:42:59.657292+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:42:59.657292+0000","last_undegraded":"2024-04-25T00:42:59.657292+0000","last_fullsized":"2024-04-25T00:42:59.657292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941489+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:42:59.941489+0000","last_peered":"2024-04-25T00:42:59.941489+0000","last_clean":"2024-04-25T00:42:59.941489+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:42:59.941489+0000","last_undegraded":"2024-04-25T00:42:59.941489+0000","last_fullsized":"2024-04-25T00:42:59.941489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014644+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:43:00.014644+0000","last_peered":"2024-04-25T00:43:00.014644+0000","last_clean":"2024-04-25T00:43:00.014644+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:43:00.014644+0000","last_undegraded":"2024-04-25T00:43:00.014644+0000","last_fullsized":"2024-04-25T00:43:00.014644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013656+0000","last_change":"2024-04-25T00:42:58.693414+0000","last_active":"2024-04-25T00:43:00.013656+0000","last_peered":"2024-04-25T00:43:00.013656+0000","last_clean":"2024-04-25T00:43:00.013656+0000","last_became_active":"2024-04-25T00:42:58.693071+0000","last_became_peered":"2024-04-25T00:42:58.693071+0000","last_unstale":"2024-04-25T00:43:00.013656+0000","last_undegraded":"2024-04-25T00:43:00.013656+0000","last_fullsized":"2024-04-25T00:43:00.013656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658896+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:42:59.658896+0000","last_peered":"2024-04-25T00:42:59.658896+0000","last_clean":"2024-04-25T00:42:59.658896+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:42:59.658896+0000","last_undegraded":"2024-04-25T00:42:59.658896+0000","last_fullsized":"2024-04-25T00:42:59.658896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662637+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:42:59.662637+0000","last_peered":"2024-04-25T00:42:59.662637+0000","last_clean":"2024-04-25T00:42:59.662637+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:42:59.662637+0000","last_undegraded":"2024-04-25T00:42:59.662637+0000","last_fullsized":"2024-04-25T00:42:59.662637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658748+0000","last_change":"2024-04-25T00:42:58.689399+0000","last_active":"2024-04-25T00:42:59.658748+0000","last_peered":"2024-04-25T00:42:59.658748+0000","last_clean":"2024-04-25T00:42:59.658748+0000","last_became_active":"2024-04-25T00:42:58.689167+0000","last_became_peered":"2024-04-25T00:42:58.689167+0000","last_unstale":"2024-04-25T00:42:59.658748+0000","last_undegraded":"2024-04-25T00:42:59.658748+0000","last_fullsized":"2024-04-25T00:42:59.658748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:06.431173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941031+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:42:59.941031+0000","last_peered":"2024-04-25T00:42:59.941031+0000","last_clean":"2024-04-25T00:42:59.941031+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:42:59.941031+0000","last_undegraded":"2024-04-25T00:42:59.941031+0000","last_fullsized":"2024-04-25T00:42:59.941031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014851+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:43:00.014851+0000","last_peered":"2024-04-25T00:43:00.014851+0000","last_clean":"2024-04-25T00:43:00.014851+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:43:00.014851+0000","last_undegraded":"2024-04-25T00:43:00.014851+0000","last_fullsized":"2024-04-25T00:43:00.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662544+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:42:59.662544+0000","last_peered":"2024-04-25T00:42:59.662544+0000","last_clean":"2024-04-25T00:42:59.662544+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:42:59.662544+0000","last_undegraded":"2024-04-25T00:42:59.662544+0000","last_fullsized":"2024-04-25T00:42:59.662544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940666+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:42:59.940666+0000","last_peered":"2024-04-25T00:42:59.940666+0000","last_clean":"2024-04-25T00:42:59.940666+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:42:59.940666+0000","last_undegraded":"2024-04-25T00:42:59.940666+0000","last_fullsized":"2024-04-25T00:42:59.940666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.940621+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:42:59.940621+0000","last_peered":"2024-04-25T00:42:59.940621+0000","last_clean":"2024-04-25T00:42:59.940621+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:42:59.940621+0000","last_undegraded":"2024-04-25T00:42:59.940621+0000","last_fullsized":"2024-04-25T00:42:59.940621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.012893+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:43:00.012893+0000","last_peered":"2024-04-25T00:43:00.012893+0000","last_clean":"2024-04-25T00:43:00.012893+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:43:00.012893+0000","last_undegraded":"2024-04-25T00:43:00.012893+0000","last_fullsized":"2024-04-25T00:43:00.012893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656623+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:42:59.656623+0000","last_peered":"2024-04-25T00:42:59.656623+0000","last_clean":"2024-04-25T00:42:59.656623+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:42:59.656623+0000","last_undegraded":"2024-04-25T00:42:59.656623+0000","last_fullsized":"2024-04-25T00:42:59.656623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663244+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:42:59.663244+0000","last_peered":"2024-04-25T00:42:59.663244+0000","last_clean":"2024-04-25T00:42:59.663244+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:42:59.663244+0000","last_undegraded":"2024-04-25T00:42:59.663244+0000","last_fullsized":"2024-04-25T00:42:59.663244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013906+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:43:00.013906+0000","last_peered":"2024-04-25T00:43:00.013906+0000","last_clean":"2024-04-25T00:43:00.013906+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:43:00.013906+0000","last_undegraded":"2024-04-25T00:43:00.013906+0000","last_fullsized":"2024-04-25T00:43:00.013906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013989+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:43:00.013989+0000","last_peered":"2024-04-25T00:43:00.013989+0000","last_clean":"2024-04-25T00:43:00.013989+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:43:00.013989+0000","last_undegraded":"2024-04-25T00:43:00.013989+0000","last_fullsized":"2024-04-25T00:43:00.013989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014621+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:43:00.014621+0000","last_peered":"2024-04-25T00:43:00.014621+0000","last_clean":"2024-04-25T00:43:00.014621+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:43:00.014621+0000","last_undegraded":"2024-04-25T00:43:00.014621+0000","last_fullsized":"2024-04-25T00:43:00.014621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657481+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:42:59.657481+0000","last_peered":"2024-04-25T00:42:59.657481+0000","last_clean":"2024-04-25T00:42:59.657481+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:42:59.657481+0000","last_undegraded":"2024-04-25T00:42:59.657481+0000","last_fullsized":"2024-04-25T00:42:59.657481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659786+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:42:59.659786+0000","last_peered":"2024-04-25T00:42:59.659786+0000","last_clean":"2024-04-25T00:42:59.659786+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:42:59.659786+0000","last_undegraded":"2024-04-25T00:42:59.659786+0000","last_fullsized":"2024-04-25T00:42:59.659786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662829+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:42:59.662829+0000","last_peered":"2024-04-25T00:42:59.662829+0000","last_clean":"2024-04-25T00:42:59.662829+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:42:59.662829+0000","last_undegraded":"2024-04-25T00:42:59.662829+0000","last_fullsized":"2024-04-25T00:42:59.662829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015191+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:43:00.015191+0000","last_peered":"2024-04-25T00:43:00.015191+0000","last_clean":"2024-04-25T00:43:00.015191+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:43:00.015191+0000","last_undegraded":"2024-04-25T00:43:00.015191+0000","last_fullsized":"2024-04-25T00:43:00.015191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014789+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:43:00.014789+0000","last_peered":"2024-04-25T00:43:00.014789+0000","last_clean":"2024-04-25T00:43:00.014789+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:43:00.014789+0000","last_undegraded":"2024-04-25T00:43:00.014789+0000","last_fullsized":"2024-04-25T00:43:00.014789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014462+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:43:00.014462+0000","last_peered":"2024-04-25T00:43:00.014462+0000","last_clean":"2024-04-25T00:43:00.014462+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:43:00.014462+0000","last_undegraded":"2024-04-25T00:43:00.014462+0000","last_fullsized":"2024-04-25T00:43:00.014462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013395+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:43:00.013395+0000","last_peered":"2024-04-25T00:43:00.013395+0000","last_clean":"2024-04-25T00:43:00.013395+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:43:00.013395+0000","last_undegraded":"2024-04-25T00:43:00.013395+0000","last_fullsized":"2024-04-25T00:43:00.013395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014992+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:43:00.014992+0000","last_peered":"2024-04-25T00:43:00.014992+0000","last_clean":"2024-04-25T00:43:00.014992+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:43:00.014992+0000","last_undegraded":"2024-04-25T00:43:00.014992+0000","last_fullsized":"2024-04-25T00:43:00.014992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662429+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:42:59.662429+0000","last_peered":"2024-04-25T00:42:59.662429+0000","last_clean":"2024-04-25T00:42:59.662429+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:42:59.662429+0000","last_undegraded":"2024-04-25T00:42:59.662429+0000","last_fullsized":"2024-04-25T00:42:59.662429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662799+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:42:59.662799+0000","last_peered":"2024-04-25T00:42:59.662799+0000","last_clean":"2024-04-25T00:42:59.662799+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:42:59.662799+0000","last_undegraded":"2024-04-25T00:42:59.662799+0000","last_fullsized":"2024-04-25T00:42:59.662799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658133+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:42:59.658133+0000","last_peered":"2024-04-25T00:42:59.658133+0000","last_clean":"2024-04-25T00:42:59.658133+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:42:59.658133+0000","last_undegraded":"2024-04-25T00:42:59.658133+0000","last_fullsized":"2024-04-25T00:42:59.658133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014619+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:43:00.014619+0000","last_peered":"2024-04-25T00:43:00.014619+0000","last_clean":"2024-04-25T00:43:00.014619+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:43:00.014619+0000","last_undegraded":"2024-04-25T00:43:00.014619+0000","last_fullsized":"2024-04-25T00:43:00.014619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659249+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:42:59.659249+0000","last_peered":"2024-04-25T00:42:59.659249+0000","last_clean":"2024-04-25T00:42:59.659249+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:42:59.659249+0000","last_undegraded":"2024-04-25T00:42:59.659249+0000","last_fullsized":"2024-04-25T00:42:59.659249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015947+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:43:00.015947+0000","last_peered":"2024-04-25T00:43:00.015947+0000","last_clean":"2024-04-25T00:43:00.015947+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:43:00.015947+0000","last_undegraded":"2024-04-25T00:43:00.015947+0000","last_fullsized":"2024-04-25T00:43:00.015947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658685+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:42:59.658685+0000","last_peered":"2024-04-25T00:42:59.658685+0000","last_clean":"2024-04-25T00:42:59.658685+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:42:59.658685+0000","last_undegraded":"2024-04-25T00:42:59.658685+0000","last_fullsized":"2024-04-25T00:42:59.658685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.016444+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:43:00.016444+0000","last_peered":"2024-04-25T00:43:00.016444+0000","last_clean":"2024-04-25T00:43:00.016444+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:43:00.016444+0000","last_undegraded":"2024-04-25T00:43:00.016444+0000","last_fullsized":"2024-04-25T00:43:00.016444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657468+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:42:59.657468+0000","last_peered":"2024-04-25T00:42:59.657468+0000","last_clean":"2024-04-25T00:42:59.657468+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:42:59.657468+0000","last_undegraded":"2024-04-25T00:42:59.657468+0000","last_fullsized":"2024-04-25T00:42:59.657468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013590+0000","last_change":"2024-04-25T00:42:58.693586+0000","last_active":"2024-04-25T00:43:00.013590+0000","last_peered":"2024-04-25T00:43:00.013590+0000","last_clean":"2024-04-25T00:43:00.013590+0000","last_became_active":"2024-04-25T00:42:58.692506+0000","last_became_peered":"2024-04-25T00:42:58.692506+0000","last_unstale":"2024-04-25T00:43:00.013590+0000","last_undegraded":"2024-04-25T00:43:00.013590+0000","last_fullsized":"2024-04-25T00:43:00.013590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658281+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:42:59.658281+0000","last_peered":"2024-04-25T00:42:59.658281+0000","last_clean":"2024-04-25T00:42:59.658281+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:42:59.658281+0000","last_undegraded":"2024-04-25T00:42:59.658281+0000","last_fullsized":"2024-04-25T00:42:59.658281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015234+0000","last_change":"2024-04-25T00:42:58.698024+0000","last_active":"2024-04-25T00:43:00.015234+0000","last_peered":"2024-04-25T00:43:00.015234+0000","last_clean":"2024-04-25T00:43:00.015234+0000","last_became_active":"2024-04-25T00:42:58.697733+0000","last_became_peered":"2024-04-25T00:42:58.697733+0000","last_unstale":"2024-04-25T00:43:00.015234+0000","last_undegraded":"2024-04-25T00:43:00.015234+0000","last_fullsized":"2024-04-25T00:43:00.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013464+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:00.013464+0000","last_peered":"2024-04-25T00:43:00.013464+0000","last_clean":"2024-04-25T00:43:00.013464+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:00.013464+0000","last_undegraded":"2024-04-25T00:43:00.013464+0000","last_fullsized":"2024-04-25T00:43:00.013464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013023+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:43:00.013023+0000","last_peered":"2024-04-25T00:43:00.013023+0000","last_clean":"2024-04-25T00:43:00.013023+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:43:00.013023+0000","last_undegraded":"2024-04-25T00:43:00.013023+0000","last_fullsized":"2024-04-25T00:43:00.013023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941039+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:42:59.941039+0000","last_peered":"2024-04-25T00:42:59.941039+0000","last_clean":"2024-04-25T00:42:59.941039+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:42:59.941039+0000","last_undegraded":"2024-04-25T00:42:59.941039+0000","last_fullsized":"2024-04-25T00:42:59.941039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013385+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:43:00.013385+0000","last_peered":"2024-04-25T00:43:00.013385+0000","last_clean":"2024-04-25T00:43:00.013385+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:43:00.013385+0000","last_undegraded":"2024-04-25T00:43:00.013385+0000","last_fullsized":"2024-04-25T00:43:00.013385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015080+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:43:00.015080+0000","last_peered":"2024-04-25T00:43:00.015080+0000","last_clean":"2024-04-25T00:43:00.015080+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:43:00.015080+0000","last_undegraded":"2024-04-25T00:43:00.015080+0000","last_fullsized":"2024-04-25T00:43:00.015080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.658309+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:42:59.658309+0000","last_peered":"2024-04-25T00:42:59.658309+0000","last_clean":"2024-04-25T00:42:59.658309+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:42:59.658309+0000","last_undegraded":"2024-04-25T00:42:59.658309+0000","last_fullsized":"2024-04-25T00:42:59.658309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656914+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:42:59.656914+0000","last_peered":"2024-04-25T00:42:59.656914+0000","last_clean":"2024-04-25T00:42:59.656914+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:42:59.656914+0000","last_undegraded":"2024-04-25T00:42:59.656914+0000","last_fullsized":"2024-04-25T00:42:59.656914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013694+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:43:00.013694+0000","last_peered":"2024-04-25T00:43:00.013694+0000","last_clean":"2024-04-25T00:43:00.013694+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:43:00.013694+0000","last_undegraded":"2024-04-25T00:43:00.013694+0000","last_fullsized":"2024-04-25T00:43:00.013694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659040+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:42:59.659040+0000","last_peered":"2024-04-25T00:42:59.659040+0000","last_clean":"2024-04-25T00:42:59.659040+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:42:59.659040+0000","last_undegraded":"2024-04-25T00:42:59.659040+0000","last_fullsized":"2024-04-25T00:42:59.659040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657076+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:42:59.657076+0000","last_peered":"2024-04-25T00:42:59.657076+0000","last_clean":"2024-04-25T00:42:59.657076+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:42:59.657076+0000","last_undegraded":"2024-04-25T00:42:59.657076+0000","last_fullsized":"2024-04-25T00:42:59.657076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014402+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:43:00.014402+0000","last_peered":"2024-04-25T00:43:00.014402+0000","last_clean":"2024-04-25T00:43:00.014402+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:43:00.014402+0000","last_undegraded":"2024-04-25T00:43:00.014402+0000","last_fullsized":"2024-04-25T00:43:00.014402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015510+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:43:00.015510+0000","last_peered":"2024-04-25T00:43:00.015510+0000","last_clean":"2024-04-25T00:43:00.015510+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:43:00.015510+0000","last_undegraded":"2024-04-25T00:43:00.015510+0000","last_fullsized":"2024-04-25T00:43:00.015510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656937+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:42:59.656937+0000","last_peered":"2024-04-25T00:42:59.656937+0000","last_clean":"2024-04-25T00:42:59.656937+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:42:59.656937+0000","last_undegraded":"2024-04-25T00:42:59.656937+0000","last_fullsized":"2024-04-25T00:42:59.656937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657625+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:42:59.657625+0000","last_peered":"2024-04-25T00:42:59.657625+0000","last_clean":"2024-04-25T00:42:59.657625+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:42:59.657625+0000","last_undegraded":"2024-04-25T00:42:59.657625+0000","last_fullsized":"2024-04-25T00:42:59.657625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941751+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:42:59.941751+0000","last_peered":"2024-04-25T00:42:59.941751+0000","last_clean":"2024-04-25T00:42:59.941751+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:42:59.941751+0000","last_undegraded":"2024-04-25T00:42:59.941751+0000","last_fullsized":"2024-04-25T00:42:59.941751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.663349+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:42:59.663349+0000","last_peered":"2024-04-25T00:42:59.663349+0000","last_clean":"2024-04-25T00:42:59.663349+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:42:59.663349+0000","last_undegraded":"2024-04-25T00:42:59.663349+0000","last_fullsized":"2024-04-25T00:42:59.663349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014387+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:43:00.014387+0000","last_peered":"2024-04-25T00:43:00.014387+0000","last_clean":"2024-04-25T00:43:00.014387+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:43:00.014387+0000","last_undegraded":"2024-04-25T00:43:00.014387+0000","last_fullsized":"2024-04-25T00:43:00.014387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013174+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:43:00.013174+0000","last_peered":"2024-04-25T00:43:00.013174+0000","last_clean":"2024-04-25T00:43:00.013174+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:43:00.013174+0000","last_undegraded":"2024-04-25T00:43:00.013174+0000","last_fullsized":"2024-04-25T00:43:00.013174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.659268+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:42:59.659268+0000","last_peered":"2024-04-25T00:42:59.659268+0000","last_clean":"2024-04-25T00:42:59.659268+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:42:59.659268+0000","last_undegraded":"2024-04-25T00:42:59.659268+0000","last_fullsized":"2024-04-25T00:42:59.659268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.656849+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:42:59.656849+0000","last_peered":"2024-04-25T00:42:59.656849+0000","last_clean":"2024-04-25T00:42:59.656849+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:42:59.656849+0000","last_undegraded":"2024-04-25T00:42:59.656849+0000","last_fullsized":"2024-04-25T00:42:59.656849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941104+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:42:59.941104+0000","last_peered":"2024-04-25T00:42:59.941104+0000","last_clean":"2024-04-25T00:42:59.941104+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:42:59.941104+0000","last_undegraded":"2024-04-25T00:42:59.941104+0000","last_fullsized":"2024-04-25T00:42:59.941104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.657161+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:42:59.657161+0000","last_peered":"2024-04-25T00:42:59.657161+0000","last_clean":"2024-04-25T00:42:59.657161+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:42:59.657161+0000","last_undegraded":"2024-04-25T00:42:59.657161+0000","last_fullsized":"2024-04-25T00:42:59.657161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941098+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:42:59.941098+0000","last_peered":"2024-04-25T00:42:59.941098+0000","last_clean":"2024-04-25T00:42:59.941098+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:42:59.941098+0000","last_undegraded":"2024-04-25T00:42:59.941098+0000","last_fullsized":"2024-04-25T00:42:59.941098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015070+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:43:00.015070+0000","last_peered":"2024-04-25T00:43:00.015070+0000","last_clean":"2024-04-25T00:43:00.015070+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:43:00.015070+0000","last_undegraded":"2024-04-25T00:43:00.015070+0000","last_fullsized":"2024-04-25T00:43:00.015070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.662834+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:42:59.662834+0000","last_peered":"2024-04-25T00:42:59.662834+0000","last_clean":"2024-04-25T00:42:59.662834+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:42:59.662834+0000","last_undegraded":"2024-04-25T00:42:59.662834+0000","last_fullsized":"2024-04-25T00:42:59.662834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014334+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:43:00.014334+0000","last_peered":"2024-04-25T00:43:00.014334+0000","last_clean":"2024-04-25T00:43:00.014334+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:43:00.014334+0000","last_undegraded":"2024-04-25T00:43:00.014334+0000","last_fullsized":"2024-04-25T00:43:00.014334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014890+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:43:00.014890+0000","last_peered":"2024-04-25T00:43:00.014890+0000","last_clean":"2024-04-25T00:43:00.014890+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:43:00.014890+0000","last_undegraded":"2024-04-25T00:43:00.014890+0000","last_fullsized":"2024-04-25T00:43:00.014890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.013948+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:43:00.013948+0000","last_peered":"2024-04-25T00:43:00.013948+0000","last_clean":"2024-04-25T00:43:00.013948+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:43:00.013948+0000","last_undegraded":"2024-04-25T00:43:00.013948+0000","last_fullsized":"2024-04-25T00:43:00.013948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015334+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:43:00.015334+0000","last_peered":"2024-04-25T00:43:00.015334+0000","last_clean":"2024-04-25T00:43:00.015334+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:43:00.015334+0000","last_undegraded":"2024-04-25T00:43:00.015334+0000","last_fullsized":"2024-04-25T00:43:00.015334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.014217+0000","last_change":"2024-04-25T00:42:58.700939+0000","last_active":"2024-04-25T00:43:00.014217+0000","last_peered":"2024-04-25T00:43:00.014217+0000","last_clean":"2024-04-25T00:43:00.014217+0000","last_became_active":"2024-04-25T00:42:58.700177+0000","last_became_peered":"2024-04-25T00:42:58.700177+0000","last_unstale":"2024-04-25T00:43:00.014217+0000","last_undegraded":"2024-04-25T00:43:00.014217+0000","last_fullsized":"2024-04-25T00:43:00.014217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:36.086387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941542+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:42:59.941542+0000","last_peered":"2024-04-25T00:42:59.941542+0000","last_clean":"2024-04-25T00:42:59.941542+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:42:59.941542+0000","last_undegraded":"2024-04-25T00:42:59.941542+0000","last_fullsized":"2024-04-25T00:42:59.941542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:42:59.941612+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:42:59.941612+0000","last_peered":"2024-04-25T00:42:59.941612+0000","last_clean":"2024-04-25T00:42:59.941612+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:42:59.941612+0000","last_undegraded":"2024-04-25T00:42:59.941612+0000","last_fullsized":"2024-04-25T00:42:59.941612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:43:00.015091+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:43:00.015091+0000","last_peered":"2024-04-25T00:43:00.015091+0000","last_clean":"2024-04-25T00:43:00.015091+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:43:00.015091+0000","last_undegraded":"2024-04-25T00:43:00.015091+0000","last_fullsized":"2024-04-25T00:43:00.015091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:43:05.682 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:43:05.682 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:43:05.682 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:43:05.682 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T00:43:05.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.824+0000 7fb58d53e640 1 Processor -- start 2024-04-25T00:43:05.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.824+0000 7fb58d53e640 1 -- start start 2024-04-25T00:43:05.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb58815bf50 0x7fb58815c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880aa0c0 0x7fb5880aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb5880ccb50 0x7fb5880cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fb58806d1b0 con 0x7fb5880ccb50 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb58806d490 con 0x7fb58815bf50 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb58806d770 con 0x7fb5880aa0c0 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb5880ccb50 0x7fb5880cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb5880ccb50 0x7fb5880cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42914/0 (socket says 172.21.15.129:42914) 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 -- 172.21.15.129:0/899258332 learned_addr learned my addr 172.21.15.129:0/899258332 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb57ffff640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880aa0c0 0x7fb5880aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb586ffd640 1 --2- 172.21.15.129:0/899258332 >> v2:172.21.15.196:3300/0 conn(0x7fb58815bf50 0x7fb58815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 -- 172.21.15.129:0/899258332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880aa0c0 msgr2=0x7fb5880aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880aa0c0 0x7fb5880aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 -- 172.21.15.129:0/899258332 >> v2:172.21.15.196:3300/0 conn(0x7fb58815bf50 msgr2=0x7fb58815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 --2- 172.21.15.129:0/899258332 >> v2:172.21.15.196:3300/0 conn(0x7fb58815bf50 0x7fb58815c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 -- 172.21.15.129:0/899258332 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58806da50 con 0x7fb5880ccb50 2024-04-25T00:43:05.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb57ffff640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880aa0c0 0x7fb5880aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5877fe640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5880ccb50 0x7fb5880cd000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb588069260 tx=0x7fb58001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335d180ffc39dc9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5867fc640 1 -- 172.21.15.129:0/899258332 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800a5780 con 0x7fb5880ccb50 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5867fc640 1 -- 172.21.15.129:0/899258332 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5800a5920 con 0x7fb5880ccb50 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb5867fc640 1 -- 172.21.15.129:0/899258332 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800a5cf0 con 0x7fb5880ccb50 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- 172.21.15.129:0/899258332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5880ccb50 msgr2=0x7fb5880cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5880ccb50 0x7fb5880cd000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb588069260 tx=0x7fb58001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- 172.21.15.129:0/899258332 shutdown_connections 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880aa0c0 0x7fb5880aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- 172.21.15.129:0/899258332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5880ccb50 0x7fb5880cd000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 --2- 172.21.15.129:0/899258332 >> v2:172.21.15.196:3300/0 conn(0x7fb58815bf50 0x7fb58815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- 172.21.15.129:0/899258332 >> 172.21.15.129:0/899258332 conn(0x7fb5880b14a0 msgr2=0x7fb5880b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- 172.21.15.129:0/899258332 shutdown_connections 2024-04-25T00:43:05.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.828+0000 7fb58d53e640 1 -- 172.21.15.129:0/899258332 wait complete. 2024-04-25T00:43:05.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 Processor -- start 2024-04-25T00:43:05.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- start start 2024-04-25T00:43:05.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880aa0c0 0x7fb588157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880ccb50 0x7fb588137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 0x7fb588146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb58806ea60 con 0x7fb58815bf50 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb58806cf70 con 0x7fb5880aa0c0 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb58806ebe0 con 0x7fb5880ccb50 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 0x7fb588146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 0x7fb588146fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42926/0 (socket says 172.21.15.129:42926) 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 -- 172.21.15.129:0/3917764480 learned_addr learned my addr 172.21.15.129:0/3917764480 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb586ffd640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880aa0c0 0x7fb588157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 -- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880ccb50 msgr2=0x7fb588137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880ccb50 0x7fb588137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 -- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880aa0c0 msgr2=0x7fb588157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880aa0c0 0x7fb588157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 -- 172.21.15.129:0/3917764480 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880b0640 con 0x7fb58815bf50 2024-04-25T00:43:05.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb5877fe640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 0x7fb588146fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb5800af040 tx=0x7fb580019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800bc020 con 0x7fb58815bf50 2024-04-25T00:43:05.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5800b8670 con 0x7fb58815bf50 2024-04-25T00:43:05.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5880b0bc0 con 0x7fb58815bf50 2024-04-25T00:43:05.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800b8a60 con 0x7fb58815bf50 2024-04-25T00:43:05.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb588155b00 con 0x7fb58815bf50 2024-04-25T00:43:05.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb588062e50 con 0x7fb58815bf50 2024-04-25T00:43:05.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.832+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5800bf020 con 0x7fb58815bf50 2024-04-25T00:43:05.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.836+0000 7fb57f7fe640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb5680678c0 0x7fb568069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:05.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.836+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5801211d0 con 0x7fb58815bf50 2024-04-25T00:43:05.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.836+0000 7fb586ffd640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb5680678c0 0x7fb568069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:05.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.836+0000 7fb586ffd640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb5680678c0 0x7fb568069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb57000b7c0 tx=0x7fb570008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:05.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:05.836+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb5800c6030 con 0x7fb58815bf50 2024-04-25T00:43:06.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.016+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb588075320 con 0x7fb58815bf50 2024-04-25T00:43:06.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb57f7fe640 1 -- 172.21.15.129:0/3917764480 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb5800165e0 con 0x7fb58815bf50 2024-04-25T00:43:06.022 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:06.023 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:43:06.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb5680678c0 msgr2=0x7fb568069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb5680678c0 0x7fb568069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb57000b7c0 tx=0x7fb570008040 comp rx=0 tx=0).stop 2024-04-25T00:43:06.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 msgr2=0x7fb588146fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 0x7fb588146fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb5800af040 tx=0x7fb580019790 comp rx=0 tx=0).stop 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 shutdown_connections 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb5680678c0 0x7fb568069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5880ccb50 0x7fb588137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58815bf50 0x7fb588146fa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 --2- 172.21.15.129:0/3917764480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880aa0c0 0x7fb588157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 >> 172.21.15.129:0/3917764480 conn(0x7fb5880b14a0 msgr2=0x7fb58815af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 shutdown_connections 2024-04-25T00:43:06.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.020+0000 7fb58d53e640 1 -- 172.21.15.129:0/3917764480 wait complete. 2024-04-25T00:43:06.040 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:43:06.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:43:06.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9cd15b640 1 Processor -- start 2024-04-25T00:43:06.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9cd15b640 1 -- start start 2024-04-25T00:43:06.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9cd15b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe9c813d9d0 0x7fe9c815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9cd15b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c80ce740 con 0x7fe9c8070690 2024-04-25T00:43:06.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9cd15b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c80ce9e0 con 0x7fe9c813dec0 2024-04-25T00:43:06.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9cd15b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c815bd30 con 0x7fe9c813d9d0 2024-04-25T00:43:06.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c759e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fe9c8070690 0x7fe9c80cf4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46748/0 (socket says 172.21.15.129:46748) 2024-04-25T00:43:06.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c759e640 1 -- 172.21.15.129:0/2967536642 learned_addr learned my addr 172.21.15.129:0/2967536642 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c6d9d640 1 --2- 172.21.15.129:0/2967536642 >> v2:172.21.15.196:3300/0 conn(0x7fe9c813d9d0 0x7fe9c815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977784558 0 0) 0x7fe9c80ce740 con 0x7fe9c8070690 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9b0004640 con 0x7fe9c8070690 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1865160391 0 0) 0x7fe9b0004640 con 0x7fe9c8070690 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 >> v2:172.21.15.196:3300/0 conn(0x7fe9c813d9d0 msgr2=0x7fe9c815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 --2- 172.21.15.129:0/2967536642 >> v2:172.21.15.196:3300/0 conn(0x7fe9c813d9d0 0x7fe9c815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 >> v1:172.21.15.196:6789/0 conn(0x7fe9c813dec0 legacy=0x7fe9c8072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c80cecc0 con 0x7fe9c8070690 2024-04-25T00:43:06.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fe9c0001a00 con 0x7fe9c8070690 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 >> v1:172.21.15.129:6789/0 conn(0x7fe9c8070690 legacy=0x7fe9c80cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0004ad0 0x7fe9b0004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b0005920 0x7fe9b0005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9b00062e0 0x7fe9b000cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c815bd30 con 0x7fe9b0005920 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b000d180 con 0x7fe9b0004ad0 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b000d380 con 0x7fe9b00062e0 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c659c640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b0005920 0x7fe9b0005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c6d9d640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0004ad0 0x7fe9b0004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c759e640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9b00062e0 0x7fe9b000cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c659c640 1 -- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9b00062e0 msgr2=0x7fe9b000cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c659c640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9b00062e0 0x7fe9b000cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c659c640 1 -- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0004ad0 msgr2=0x7fe9b0004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c659c640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0004ad0 0x7fe9b0004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c659c640 1 -- 172.21.15.129:0/2967536642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b0004640 con 0x7fe9b0005920 2024-04-25T00:43:06.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.156+0000 7fe9c759e640 1 -- 172.21.15.129:0/2967536642 reap_dead start 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b0005920 0x7fe9b0005da0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc001780 tx=0x7fe9bc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4f6e134a4b92de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0013a0 con 0x7fe9b0005920 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc0a9070 con 0x7fe9b0005920 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c5d9b640 1 -- 172.21.15.129:0/2967536642 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc003600 con 0x7fe9b0005920 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- 172.21.15.129:0/2967536642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b0005920 msgr2=0x7fe9b0005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b0005920 0x7fe9b0005da0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc001780 tx=0x7fe9bc01c420 comp rx=0 tx=0).stop 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- 172.21.15.129:0/2967536642 shutdown_connections 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/2967536642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9b0005920 0x7fe9b0005da0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- 172.21.15.129:0/2967536642 >> 172.21.15.129:0/2967536642 conn(0x7fe9c80b2fe0 msgr2=0x7fe9c80b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- 172.21.15.129:0/2967536642 shutdown_connections 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- 172.21.15.129:0/2967536642 wait complete. 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 Processor -- start 2024-04-25T00:43:06.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- start start 2024-04-25T00:43:06.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0005920 0x7fe9c8132310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 0x7fe9c8131330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9c812fef0 0x7fe9c8130370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c813e340 con 0x7fe9c8130eb0 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c80cf030 con 0x7fe9b0005920 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9cd15b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c813e4c0 con 0x7fe9c812fef0 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 0x7fe9c8131330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 0x7fe9c8131330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42932/0 (socket says 172.21.15.129:42932) 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 -- 172.21.15.129:0/4279433113 learned_addr learned my addr 172.21.15.129:0/4279433113 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c759e640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9c812fef0 0x7fe9c8130370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c6d9d640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0005920 0x7fe9c8132310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 -- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9c812fef0 msgr2=0x7fe9c8130370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9c812fef0 0x7fe9c8130370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 -- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0005920 msgr2=0x7fe9c8132310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0005920 0x7fe9c8132310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 -- 172.21.15.129:0/4279433113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c812eec0 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c6d9d640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0005920 0x7fe9c8132310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.160+0000 7fe9c659c640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 0x7fe9c8131330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc01c250 tx=0x7fe9b0006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0bb070 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc007250 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c812f130 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc007640 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c812f6f0 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c8062e50 con 0x7fe9c8130eb0 2024-04-25T00:43:06.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9bc0be070 con 0x7fe9c8130eb0 2024-04-25T00:43:06.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9af7fe640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9b40678f0 0x7fe9b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe9bc11edf0 con 0x7fe9c8130eb0 2024-04-25T00:43:06.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9c6d9d640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9b40678f0 0x7fe9b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.164+0000 7fe9c6d9d640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9b40678f0 0x7fe9b4069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8067040 tx=0x7fe9b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.168+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe9bc016130 con 0x7fe9c8130eb0 2024-04-25T00:43:06.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.316+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9c8075320 con 0x7fe9c8130eb0 2024-04-25T00:43:06.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9af7fe640 1 -- 172.21.15.129:0/4279433113 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7fe9bc0ef9c0 con 0x7fe9c8130eb0 2024-04-25T00:43:06.323 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:06.323 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:59.648406+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:43:06.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9b40678f0 msgr2=0x7fe9b4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9b40678f0 0x7fe9b4069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8067040 tx=0x7fe9b8008040 comp rx=0 tx=0).stop 2024-04-25T00:43:06.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 msgr2=0x7fe9c8131330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 0x7fe9c8131330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc01c250 tx=0x7fe9b0006950 comp rx=0 tx=0).stop 2024-04-25T00:43:06.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 shutdown_connections 2024-04-25T00:43:06.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9b40678f0 0x7fe9b4069db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9c812fef0 0x7fe9c8130370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9c8130eb0 0x7fe9c8131330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 --2- 172.21.15.129:0/4279433113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9b0005920 0x7fe9c8132310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 >> 172.21.15.129:0/4279433113 conn(0x7fe9c80b2fe0 msgr2=0x7fe9c8074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 shutdown_connections 2024-04-25T00:43:06.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.320+0000 7fe9cd15b640 1 -- 172.21.15.129:0/4279433113 wait complete. 2024-04-25T00:43:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:43:06.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:43:06.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.452+0000 7fe1ab9b8640 1 Processor -- start 2024-04-25T00:43:06.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1ab9b8640 1 -- start start 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1ab9b8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1a40af190 0x7fe1a4169010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1ab9b8640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a40b2f10 con 0x7fe1a40b4660 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1ab9b8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a40b3180 con 0x7fe1a40cffc0 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1ab9b8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a40b3460 con 0x7fe1a40af190 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1a40af190 0x7fe1a4169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1a40af190 0x7fe1a4169010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42936/0 (socket says 172.21.15.129:42936) 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 -- 172.21.15.129:0/2758069389 learned_addr learned my addr 172.21.15.129:0/2758069389 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:06.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 -- 172.21.15.129:0/2758069389 >> v1:172.21.15.196:6789/0 conn(0x7fe1a40cffc0 legacy=0x7fe1a415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 -- 172.21.15.129:0/2758069389 >> v1:172.21.15.129:6789/0 conn(0x7fe1a40b4660 legacy=0x7fe1a40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 -- 172.21.15.129:0/2758069389 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a40b3740 con 0x7fe1a40af190 2024-04-25T00:43:06.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe19bfff640 1 -- 172.21.15.129:0/2758069389 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384408791 0 0) 0x7fe1a40b2f10 con 0x7fe1a40b4660 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe1a9f55640 1 --2- 172.21.15.129:0/2758069389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40af190 0x7fe1a4169010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe194009870 tx=0x7fe19401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8fa3688c245754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe19bfff640 1 -- 172.21.15.129:0/2758069389 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1940ab430 con 0x7fe1a40af190 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe19bfff640 1 -- 172.21.15.129:0/2758069389 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1940b0070 con 0x7fe1a40af190 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.456+0000 7fe19bfff640 1 -- 172.21.15.129:0/2758069389 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1940ab7f0 con 0x7fe1a40af190 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/2758069389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40af190 msgr2=0x7fe1a4169010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/2758069389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40af190 0x7fe1a4169010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe194009870 tx=0x7fe19401c560 comp rx=0 tx=0).stop 2024-04-25T00:43:06.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/2758069389 shutdown_connections 2024-04-25T00:43:06.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/2758069389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40af190 0x7fe1a4169010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/2758069389 >> 172.21.15.129:0/2758069389 conn(0x7fe1a4071ac0 msgr2=0x7fe1a4071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/2758069389 shutdown_connections 2024-04-25T00:43:06.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/2758069389 wait complete. 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 Processor -- start 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- start start 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a40af190 0x7fe1a415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 0x7fe1a414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40cffc0 0x7fe1a414af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a40d0520 con 0x7fe1a40b4660 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a40d03a0 con 0x7fe1a40af190 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a416bd70 con 0x7fe1a40cffc0 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 0x7fe1a414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 0x7fe1a414fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42944/0 (socket says 172.21.15.129:42944) 2024-04-25T00:43:06.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 -- 172.21.15.129:0/572864573 learned_addr learned my addr 172.21.15.129:0/572864573 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a9754640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a40af190 0x7fe1a415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 -- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40cffc0 msgr2=0x7fe1a414af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40cffc0 0x7fe1a414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 -- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a40af190 msgr2=0x7fe1a415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a40af190 0x7fe1a415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 -- 172.21.15.129:0/572864573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a4158310 con 0x7fe1a40b4660 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a9754640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a40af190 0x7fe1a415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1a8f53640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 0x7fe1a414fce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c810 tx=0x7fe1a000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0003db0 con 0x7fe1a40b4660 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a0005070 con 0x7fe1a40b4660 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a41585b0 con 0x7fe1a40b4660 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0005460 con 0x7fe1a40b4660 2024-04-25T00:43:06.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a4157000 con 0x7fe1a40b4660 2024-04-25T00:43:06.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.460+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a4062e50 con 0x7fe1a40b4660 2024-04-25T00:43:06.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.464+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1a0005600 con 0x7fe1a40b4660 2024-04-25T00:43:06.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.464+0000 7fe199ffb640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1880678c0 0x7fe188069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.464+0000 7fe1a9754640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1880678c0 0x7fe188069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.464+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe1a0085c00 con 0x7fe1a40b4660 2024-04-25T00:43:06.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.464+0000 7fe1a9754640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1880678c0 0x7fe188069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe19c000f70 tx=0x7fe19c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.468+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe1a008c080 con 0x7fe1a40b4660 2024-04-25T00:43:06.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1a4075320 con 0x7fe1a40b4660 2024-04-25T00:43:06.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe199ffb640 1 -- 172.21.15.129:0/572864573 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe1a00567d0 con 0x7fe1a40b4660 2024-04-25T00:43:06.631 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:43:06.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1880678c0 msgr2=0x7fe188069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1880678c0 0x7fe188069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe19c000f70 tx=0x7fe19c009040 comp rx=0 tx=0).stop 2024-04-25T00:43:06.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 msgr2=0x7fe1a414fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 0x7fe1a414fce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c810 tx=0x7fe1a000ccd0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 shutdown_connections 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1880678c0 0x7fe188069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40cffc0 0x7fe1a414af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a40b4660 0x7fe1a414fce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 --2- 172.21.15.129:0/572864573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a40af190 0x7fe1a415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 >> 172.21.15.129:0/572864573 conn(0x7fe1a4071ac0 msgr2=0x7fe1a4154990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 shutdown_connections 2024-04-25T00:43:06.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.628+0000 7fe1ab9b8640 1 -- 172.21.15.129:0/572864573 wait complete. 2024-04-25T00:43:06.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:43:06.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 Processor -- start 2024-04-25T00:43:06.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 -- start start 2024-04-25T00:43:06.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f48cc15b0d0 0x7f48cc0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 0x7f48cc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0cf880 con 0x7f48cc06d890 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc0d0490 con 0x7f48cc15b0d0 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.764+0000 7f48d2755640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc0cfb60 con 0x7f48cc0cf0a0 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 0x7f48cc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 0x7f48cc06d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53856/0 (socket says 172.21.15.129:53856) 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 -- 172.21.15.129:0/2762895810 learned_addr learned my addr 172.21.15.129:0/2762895810 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48cbfff640 1 --2- 172.21.15.129:0/2762895810 >> v2:172.21.15.196:3300/0 conn(0x7f48cc15b0d0 0x7f48cc0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 -- 172.21.15.129:0/2762895810 >> v2:172.21.15.196:3300/0 conn(0x7f48cc15b0d0 msgr2=0x7f48cc0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:06.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 --2- 172.21.15.129:0/2762895810 >> v2:172.21.15.196:3300/0 conn(0x7f48cc15b0d0 0x7f48cc0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 -- 172.21.15.129:0/2762895810 >> v1:172.21.15.196:6789/0 conn(0x7f48cc06d890 legacy=0x7f48cc15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 -- 172.21.15.129:0/2762895810 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0cfe40 con 0x7f48cc0cf0a0 2024-04-25T00:43:06.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48cbfff640 1 --2- 172.21.15.129:0/2762895810 >> v2:172.21.15.196:3300/0 conn(0x7f48cc15b0d0 0x7f48cc0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:43:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48c3fff640 1 --2- 172.21.15.129:0/2762895810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 0x7f48cc06d350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feede3edec6a846d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48cb7fe640 1 -- 172.21.15.129:0/2762895810 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40b0070 con 0x7f48cc0cf0a0 2024-04-25T00:43:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48cb7fe640 1 -- 172.21.15.129:0/2762895810 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b40ab600 con 0x7f48cc0cf0a0 2024-04-25T00:43:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.768+0000 7f48cb7fe640 1 -- 172.21.15.129:0/2762895810 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40ab9d0 con 0x7f48cc0cf0a0 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/2762895810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 msgr2=0x7f48cc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 --2- 172.21.15.129:0/2762895810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 0x7f48cc06d350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b401c4d0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/2762895810 shutdown_connections 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 --2- 172.21.15.129:0/2762895810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc0cf0a0 0x7f48cc06d350 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 --2- 172.21.15.129:0/2762895810 >> v2:172.21.15.196:3300/0 conn(0x7f48cc15b0d0 0x7f48cc0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/2762895810 >> 172.21.15.129:0/2762895810 conn(0x7f48cc071ac0 msgr2=0x7f48cc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/2762895810 shutdown_connections 2024-04-25T00:43:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/2762895810 wait complete. 2024-04-25T00:43:06.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 Processor -- start 2024-04-25T00:43:06.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- start start 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc06d890 0x7f48cc13a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc0cf0a0 0x7f48cc15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 0x7f48cc138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc168000 con 0x7f48cc15b0d0 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc167e80 con 0x7f48cc0cf0a0 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc168180 con 0x7f48cc06d890 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 0x7f48cc138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 0x7f48cc138300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42960/0 (socket says 172.21.15.129:42960) 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 -- 172.21.15.129:0/322775565 learned_addr learned my addr 172.21.15.129:0/322775565 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48cbfff640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc06d890 0x7f48cc13a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48c3fff640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc0cf0a0 0x7f48cc15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 -- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc06d890 msgr2=0x7f48cc13a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc06d890 0x7f48cc13a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 -- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc0cf0a0 msgr2=0x7f48cc15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc0cf0a0 0x7f48cc15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 -- 172.21.15.129:0/322775565 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc15dd70 con 0x7f48cc15b0d0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d0cf2640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 0x7f48cc138300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c760 tx=0x7f48c400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4003b80 con 0x7f48cc15b0d0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c4003d20 con 0x7f48cc15b0d0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc15e050 con 0x7f48cc15b0d0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400aa30 con 0x7f48cc15b0d0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.772+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc15e610 con 0x7f48cc15b0d0 2024-04-25T00:43:06.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.776+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc062e50 con 0x7f48cc15b0d0 2024-04-25T00:43:06.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.776+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48c400abd0 con 0x7f48cc15b0d0 2024-04-25T00:43:06.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.776+0000 7f48c97fa640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f48ac0678c0 0x7f48ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:06.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.776+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48c4085d50 con 0x7f48cc15b0d0 2024-04-25T00:43:06.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.776+0000 7f48cbfff640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f48ac0678c0 0x7f48ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:06.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.776+0000 7f48cbfff640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f48ac0678c0 0x7f48ac069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009580 tx=0x7f48bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:06.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.780+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f48c40566e0 con 0x7f48cc15b0d0 2024-04-25T00:43:06.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48cc071260 con 0x7f48cc15b0d0 2024-04-25T00:43:06.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48c97fa640 1 -- 172.21.15.129:0/322775565 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f48c4055fe0 con 0x7f48cc15b0d0 2024-04-25T00:43:06.932 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f48ac0678c0 msgr2=0x7f48ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f48ac0678c0 0x7f48ac069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009580 tx=0x7f48bc008040 comp rx=0 tx=0).stop 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 msgr2=0x7f48cc138300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 0x7f48cc138300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c760 tx=0x7f48c400cc20 comp rx=0 tx=0).stop 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 shutdown_connections 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f48ac0678c0 0x7f48ac069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48cc15b0d0 0x7f48cc138300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc0cf0a0 0x7f48cc15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 --2- 172.21.15.129:0/322775565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48cc06d890 0x7f48cc13a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:06.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 >> 172.21.15.129:0/322775565 conn(0x7f48cc071ac0 msgr2=0x7f48cc0b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:06.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 shutdown_connections 2024-04-25T00:43:06.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:06.928+0000 7f48d2755640 1 -- 172.21.15.129:0/322775565 wait complete. 2024-04-25T00:43:06.949 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:43:07.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 Processor -- start 2024-04-25T00:43:07.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 -- start start 2024-04-25T00:43:07.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010070da0 0x7fb010162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb01015e7b0 0x7fb01015ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb01015f1a0 con 0x7fb010141210 2024-04-25T00:43:07.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb0100aee10 con 0x7fb01015e7b0 2024-04-25T00:43:07.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb016ecc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0101415f0 con 0x7fb010070da0 2024-04-25T00:43:07.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb01015e7b0 0x7fb01015ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb00ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010070da0 0x7fb010162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb01015e7b0 0x7fb01015ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:46184/0 (socket says 172.21.15.129:46184) 2024-04-25T00:43:07.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 -- 172.21.15.129:0/1479435239 learned_addr learned my addr 172.21.15.129:0/1479435239 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:07.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 -- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010070da0 msgr2=0x7fb010162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 --2- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010070da0 0x7fb010162970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 -- 172.21.15.129:0/1479435239 >> v1:172.21.15.196:6789/0 conn(0x7fb010141210 legacy=0x7fb010070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb015469640 1 -- 172.21.15.129:0/1479435239 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01015f450 con 0x7fb01015e7b0 2024-04-25T00:43:07.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.056+0000 7fb00ffff640 1 --2- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010070da0 0x7fb010162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:43:07.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb015469640 1 --2- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb01015ec60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb008009e80 tx=0x7fb00801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7d0fa9e52de869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb00f7fe640 1 -- 172.21.15.129:0/1479435239 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0080a5810 con 0x7fb01015e7b0 2024-04-25T00:43:07.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb00f7fe640 1 -- 172.21.15.129:0/1479435239 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0080a59b0 con 0x7fb01015e7b0 2024-04-25T00:43:07.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb00f7fe640 1 -- 172.21.15.129:0/1479435239 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0080a5d80 con 0x7fb01015e7b0 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 -- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 msgr2=0x7fb01015ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 --2- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb01015ec60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb008009e80 tx=0x7fb00801c950 comp rx=0 tx=0).stop 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 -- 172.21.15.129:0/1479435239 shutdown_connections 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 --2- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb01015ec60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 --2- 172.21.15.129:0/1479435239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010070da0 0x7fb010162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 -- 172.21.15.129:0/1479435239 >> 172.21.15.129:0/1479435239 conn(0x7fb010067730 msgr2=0x7fb0100b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 -- 172.21.15.129:0/1479435239 shutdown_connections 2024-04-25T00:43:07.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 -- 172.21.15.129:0/1479435239 wait complete. 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 Processor -- start 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.060+0000 7fb016ecc640 1 -- start start 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 0x7fb01013a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010141210 0x7fb01013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb0100cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0100cc610 con 0x7fb010070da0 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb010162450 con 0x7fb01015e7b0 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0100cc790 con 0x7fb010141210 2024-04-25T00:43:07.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 0x7fb01013a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 0x7fb01013a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42964/0 (socket says 172.21.15.129:42964) 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 -- 172.21.15.129:0/4075170314 learned_addr learned my addr 172.21.15.129:0/4075170314 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00ffff640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010141210 0x7fb01013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 -- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010141210 msgr2=0x7fb01013e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb015469640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb0100cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010141210 0x7fb01013e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 -- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 msgr2=0x7fb0100cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb0100cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 -- 172.21.15.129:0/4075170314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01013eed0 con 0x7fb010070da0 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb014c68640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 0x7fb01013a5c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb00400c450 tx=0x7fb00400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004016020 con 0x7fb010070da0 2024-04-25T00:43:07.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0040052d0 con 0x7fb010070da0 2024-04-25T00:43:07.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01013f1b0 con 0x7fb010070da0 2024-04-25T00:43:07.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004007260 con 0x7fb010070da0 2024-04-25T00:43:07.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb010141770 con 0x7fb010070da0 2024-04-25T00:43:07.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb010062e50 con 0x7fb010070da0 2024-04-25T00:43:07.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb004007400 con 0x7fb010070da0 2024-04-25T00:43:07.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00d7fa640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faffc0678c0 0x7faffc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00ffff640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faffc0678c0 0x7faffc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb0040855c0 con 0x7fb010070da0 2024-04-25T00:43:07.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.064+0000 7fb00ffff640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faffc0678c0 0x7faffc069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb000009580 tx=0x7fb000008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.068+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb004020020 con 0x7fb010070da0 2024-04-25T00:43:07.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.216+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb010075320 con 0x7fb010070da0 2024-04-25T00:43:07.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.216+0000 7fb00d7fa640 1 -- 172.21.15.129:0/4075170314 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb004005480 con 0x7fb010070da0 2024-04-25T00:43:07.222 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:43:07.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faffc0678c0 msgr2=0x7faffc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faffc0678c0 0x7faffc069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb000009580 tx=0x7fb000008040 comp rx=0 tx=0).stop 2024-04-25T00:43:07.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 msgr2=0x7fb01013a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 0x7fb01013a5c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb00400c450 tx=0x7fb00400c910 comp rx=0 tx=0).stop 2024-04-25T00:43:07.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 shutdown_connections 2024-04-25T00:43:07.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faffc0678c0 0x7faffc069d80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb010141210 0x7fb01013e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb01015e7b0 0x7fb0100cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 --2- 172.21.15.129:0/4075170314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb010070da0 0x7fb01013a5c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 >> 172.21.15.129:0/4075170314 conn(0x7fb010067730 msgr2=0x7fb0100b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 shutdown_connections 2024-04-25T00:43:07.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.220+0000 7fb016ecc640 1 -- 172.21.15.129:0/4075170314 wait complete. 2024-04-25T00:43:07.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:43:07.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.340+0000 7f177dbf9640 1 Processor -- start 2024-04-25T00:43:07.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.340+0000 7f177dbf9640 1 -- start start 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f17780b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f17780b6a20 0x7f1778141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177815eac0 con 0x7f17781441b0 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f17780b6e80 con 0x7f17780b6a20 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f177815f6d0 con 0x7f1778144590 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f17780b6a20 0x7f1778141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f17777fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f17781441b0 0x7f17781627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46768/0 (socket says 172.21.15.129:46768) 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f17780b6a20 0x7f1778141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42972/0 (socket says 172.21.15.129:42972) 2024-04-25T00:43:07.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f17777fe640 1 -- 172.21.15.129:0/2889622780 learned_addr learned my addr 172.21.15.129:0/2889622780 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:07.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1776ffd640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f17780b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 -- 172.21.15.129:0/2889622780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 msgr2=0x7f17780b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f17780b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 -- 172.21.15.129:0/2889622780 >> v1:172.21.15.129:6789/0 conn(0x7f17781441b0 legacy=0x7f17781627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f17767fc640 1 -- 172.21.15.129:0/2889622780 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925415186 0 0) 0x7f177815eac0 con 0x7f17781441b0 2024-04-25T00:43:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 -- 172.21.15.129:0/2889622780 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177815eda0 con 0x7f17780b6a20 2024-04-25T00:43:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1776ffd640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f17780b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f1777fff640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17780b6a20 0x7f1778141ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17680097c0 tx=0x7f176801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec4ac4489017c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f17767fc640 1 -- 172.21.15.129:0/2889622780 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17680a7070 con 0x7f17780b6a20 2024-04-25T00:43:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f17767fc640 1 -- 172.21.15.129:0/2889622780 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17680b0070 con 0x7f17780b6a20 2024-04-25T00:43:07.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f17767fc640 1 -- 172.21.15.129:0/2889622780 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17680ab890 con 0x7f17780b6a20 2024-04-25T00:43:07.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- 172.21.15.129:0/2889622780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17780b6a20 msgr2=0x7f1778141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17780b6a20 0x7f1778141ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17680097c0 tx=0x7f176801c470 comp rx=0 tx=0).stop 2024-04-25T00:43:07.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- 172.21.15.129:0/2889622780 shutdown_connections 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17780b6a20 0x7f1778141ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 --2- 172.21.15.129:0/2889622780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f17780b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- 172.21.15.129:0/2889622780 >> 172.21.15.129:0/2889622780 conn(0x7f17780b52d0 msgr2=0x7f17780b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- 172.21.15.129:0/2889622780 shutdown_connections 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.344+0000 7f177dbf9640 1 -- 172.21.15.129:0/2889622780 wait complete. 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 Processor -- start 2024-04-25T00:43:07.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- start start 2024-04-25T00:43:07.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17780b6a20 0x7f177813c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 0x7f177813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f177813f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1778143c10 con 0x7f17781441b0 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1778143a90 con 0x7f17780b6a20 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1778142800 con 0x7f1778144590 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 0x7f177813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 0x7f177813ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42988/0 (socket says 172.21.15.129:42988) 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 -- 172.21.15.129:0/317729648 learned_addr learned my addr 172.21.15.129:0/317729648 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f17777fe640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17780b6a20 0x7f177813c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1777fff640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f177813f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 -- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 msgr2=0x7f177813f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f177813f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 -- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17780b6a20 msgr2=0x7f177813c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17780b6a20 0x7f177813c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 -- 172.21.15.129:0/317729648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177813fb40 con 0x7f17781441b0 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1777fff640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f177813f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f1776ffd640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 0x7f177813ada0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f176c00c820 tx=0x7f176c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c015070 con 0x7f17781441b0 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c007a30 con 0x7f17781441b0 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177813fe20 con 0x7f17781441b0 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c007e40 con 0x7f17781441b0 2024-04-25T00:43:07.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17781403e0 con 0x7f17781441b0 2024-04-25T00:43:07.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.348+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1778062e50 con 0x7f17781441b0 2024-04-25T00:43:07.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.352+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f176c018070 con 0x7f17781441b0 2024-04-25T00:43:07.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.352+0000 7f175ffff640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17640678c0 0x7f1764069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.352+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f176c085c10 con 0x7f17781441b0 2024-04-25T00:43:07.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.352+0000 7f17777fe640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17640678c0 0x7f1764069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.352+0000 7f17777fe640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17640678c0 0x7f1764069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1770007fd0 tx=0x7f1770007ee0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.356+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f176c056750 con 0x7f17781441b0 2024-04-25T00:43:07.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1778075320 con 0x7f17781441b0 2024-04-25T00:43:07.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f175ffff640 1 -- 172.21.15.129:0/317729648 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f176c056050 con 0x7f17781441b0 2024-04-25T00:43:07.541 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:07.541 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:42:58.001381+0000","modified":"2024-04-25T00:43:01.006515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-25T00:43:07.543 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T00:43:07.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17640678c0 msgr2=0x7f1764069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17640678c0 0x7f1764069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1770007fd0 tx=0x7f1770007ee0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 msgr2=0x7f177813ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 0x7f177813ada0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f176c00c820 tx=0x7f176c00cce0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 shutdown_connections 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17640678c0 0x7f1764069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1778144590 0x7f177813f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17781441b0 0x7f177813ada0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 --2- 172.21.15.129:0/317729648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17780b6a20 0x7f177813c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 >> 172.21.15.129:0/317729648 conn(0x7f17780b52d0 msgr2=0x7f17780b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 shutdown_connections 2024-04-25T00:43:07.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.536+0000 7f177dbf9640 1 -- 172.21.15.129:0/317729648 wait complete. 2024-04-25T00:43:07.557 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:43:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 Processor -- start 2024-04-25T00:43:07.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 -- start start 2024-04-25T00:43:07.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09300ccdb0 0x7f09300723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f093015c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0930072aa0 con 0x7f09300cd190 2024-04-25T00:43:07.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0930070690 con 0x7f09300ccdb0 2024-04-25T00:43:07.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f0935792640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09300708e0 con 0x7f093015c7f0 2024-04-25T00:43:07.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092e7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f093015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092e7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f093015c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:53912/0 (socket says 172.21.15.129:53912) 2024-04-25T00:43:07.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092effd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f09300ccdb0 0x7f09300723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092e7fc640 1 -- 172.21.15.129:0/352241649 learned_addr learned my addr 172.21.15.129:0/352241649 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:07.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092effd640 1 -- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 msgr2=0x7f093015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092effd640 1 --2- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f093015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092effd640 1 -- 172.21.15.129:0/352241649 >> v1:172.21.15.196:6789/0 conn(0x7f09300cd190 legacy=0x7f0930158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.668+0000 7f092effd640 1 -- 172.21.15.129:0/352241649 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0930070bc0 con 0x7f09300ccdb0 2024-04-25T00:43:07.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f092effd640 1 --2- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300ccdb0 0x7f09300723e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09200097c0 tx=0x7f092001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109dc3ea08dbb256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f092dffb640 1 -- 172.21.15.129:0/352241649 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200b0070 con 0x7f09300ccdb0 2024-04-25T00:43:07.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f092dffb640 1 -- 172.21.15.129:0/352241649 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09200ab600 con 0x7f09300ccdb0 2024-04-25T00:43:07.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f092dffb640 1 -- 172.21.15.129:0/352241649 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200ab9d0 con 0x7f09300ccdb0 2024-04-25T00:43:07.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 -- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300ccdb0 msgr2=0x7f09300723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 --2- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300ccdb0 0x7f09300723e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09200097c0 tx=0x7f092001c410 comp rx=0 tx=0).stop 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 -- 172.21.15.129:0/352241649 shutdown_connections 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 --2- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f093015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 --2- 172.21.15.129:0/352241649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300ccdb0 0x7f09300723e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 -- 172.21.15.129:0/352241649 >> 172.21.15.129:0/352241649 conn(0x7f09300b14a0 msgr2=0x7f09300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 -- 172.21.15.129:0/352241649 shutdown_connections 2024-04-25T00:43:07.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.672+0000 7f0935792640 1 -- 172.21.15.129:0/352241649 wait complete. 2024-04-25T00:43:07.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 Processor -- start 2024-04-25T00:43:07.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- start start 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 0x7f09300af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300cd190 0x7f093014cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f0930153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0930166e20 con 0x7f09300ccdb0 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0930166ca0 con 0x7f09300cd190 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0930166fa0 con 0x7f093015c7f0 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 0x7f09300af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 0x7f09300af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43004/0 (socket says 172.21.15.129:43004) 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 -- 172.21.15.129:0/321097929 learned_addr learned my addr 172.21.15.129:0/321097929 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:07.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092e7fc640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300cd190 0x7f093014cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092f7fe640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f0930153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 -- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 msgr2=0x7f0930153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f0930153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 -- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300cd190 msgr2=0x7f093014cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300cd190 0x7f093014cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 -- 172.21.15.129:0/321097929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093014d2c0 con 0x7f09300ccdb0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092e7fc640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300cd190 0x7f093014cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092f7fe640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f0930153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f092effd640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 0x7f09300af090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f092001a040 tx=0x7f092001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200b0050 con 0x7f09300ccdb0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09200bc070 con 0x7f09300ccdb0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0930154770 con 0x7f09300ccdb0 2024-04-25T00:43:07.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200b5a70 con 0x7f09300ccdb0 2024-04-25T00:43:07.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0930154d30 con 0x7f09300ccdb0 2024-04-25T00:43:07.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930062e50 con 0x7f09300ccdb0 2024-04-25T00:43:07.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f09200b5c10 con 0x7f09300ccdb0 2024-04-25T00:43:07.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.676+0000 7f09177fe640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f091c0678c0 0x7f091c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.680+0000 7f092e7fc640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f091c0678c0 0x7f091c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.680+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0920120560 con 0x7f09300ccdb0 2024-04-25T00:43:07.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.680+0000 7f092e7fc640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f091c0678c0 0x7f091c069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0930155640 tx=0x7f0924008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.684+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0920016090 con 0x7f09300ccdb0 2024-04-25T00:43:07.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.844+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f093015a960 con 0x7f09300ccdb0 2024-04-25T00:43:07.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.844+0000 7f09177fe640 1 -- 172.21.15.129:0/321097929 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f09200f1130 con 0x7f09300ccdb0 2024-04-25T00:43:07.850 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:07.850 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:59.648406+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:43:07.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f091c0678c0 msgr2=0x7f091c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f091c0678c0 0x7f091c069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0930155640 tx=0x7f0924008040 comp rx=0 tx=0).stop 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 msgr2=0x7f09300af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 0x7f09300af090 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f092001a040 tx=0x7f092001ac50 comp rx=0 tx=0).stop 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 shutdown_connections 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f091c0678c0 0x7f091c069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f093015c7f0 0x7f0930153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09300cd190 0x7f093014cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 --2- 172.21.15.129:0/321097929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09300ccdb0 0x7f09300af090 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 >> 172.21.15.129:0/321097929 conn(0x7f09300b14a0 msgr2=0x7f09300b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 shutdown_connections 2024-04-25T00:43:07.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.848+0000 7f0935792640 1 -- 172.21.15.129:0/321097929 wait complete. 2024-04-25T00:43:07.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:43:07.870 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:43:07.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 Processor -- start 2024-04-25T00:43:07.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 -- start start 2024-04-25T00:43:07.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 0x7f39bc0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39bc06fa80 0x7f39bc0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39bc0ccb80 con 0x7f39bc15c170 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f39bc0cc930 con 0x7f39bc06fa80 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39c17b6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39bc0cd4b0 con 0x7f39bc15bd90 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39b3fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f39bc15c170 0x7f39bc06f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46770/0 (socket says 172.21.15.129:46770) 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39b3fff640 1 -- 172.21.15.129:0/2390234533 learned_addr learned my addr 172.21.15.129:0/2390234533 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39baffd640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 0x7f39bc0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317375492 0 0) 0x7f39bc0ccb80 con 0x7f39bc15c170 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/2390234533 >> v2:172.21.15.196:3300/0 conn(0x7f39bc06fa80 0x7f39bc0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39a8003530 con 0x7f39bc15c170 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1982324417 0 0) 0x7f39a8003530 con 0x7f39bc15c170 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 msgr2=0x7f39bc0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 0x7f39bc0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 >> v2:172.21.15.196:3300/0 conn(0x7f39bc06fa80 msgr2=0x7f39bc0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 --2- 172.21.15.129:0/2390234533 >> v2:172.21.15.196:3300/0 conn(0x7f39bc06fa80 0x7f39bc0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0cce60 con 0x7f39bc15c170 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f39a4001680 con 0x7f39bc15c170 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 >> v1:172.21.15.129:6789/0 conn(0x7f39bc15c170 legacy=0x7f39bc06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80039c0 0x7f39a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 0x7f39a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a8004d50 0x7f39a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cc930 con 0x7f39a8004d50 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cd4b0 con 0x7f39a80039c0 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39a800b9d0 con 0x7f39bc15bd90 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a8004d50 0x7f39a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39baffd640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80039c0 0x7f39a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 -- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 msgr2=0x7f39a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc15bd90 0x7f39a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 -- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80039c0 msgr2=0x7f39a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:07.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80039c0 0x7f39a8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39bb7fe640 1 -- 172.21.15.129:0/2390234533 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a8003530 con 0x7f39a8004d50 2024-04-25T00:43:07.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.988+0000 7f39baffd640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39a80039c0 0x7f39a8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a8004d50 0x7f39a80051d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009fc0 tx=0x7f39b401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91820cba1c1d3c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40b2070 con 0x7f39a8004d50 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b40a5cd0 con 0x7f39a8004d50 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39ba7fc640 1 -- 172.21.15.129:0/2390234533 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4009c10 con 0x7f39a8004d50 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- 172.21.15.129:0/2390234533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a8004d50 msgr2=0x7f39a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 --2- 172.21.15.129:0/2390234533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a8004d50 0x7f39a80051d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009fc0 tx=0x7f39b401ccc0 comp rx=0 tx=0).stop 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39bb7fe640 1 -- 172.21.15.129:0/2390234533 reap_dead start 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- 172.21.15.129:0/2390234533 shutdown_connections 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- 172.21.15.129:0/2390234533 >> 172.21.15.129:0/2390234533 conn(0x7f39bc067730 msgr2=0x7f39bc0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- 172.21.15.129:0/2390234533 shutdown_connections 2024-04-25T00:43:07.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- 172.21.15.129:0/2390234533 wait complete. 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 Processor -- start 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- start start 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a8004d50 0x7f39bc154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0a9480 0x7f39bc144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 0x7f39bc0aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cf060 con 0x7f39bc0aab00 2024-04-25T00:43:07.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cd940 con 0x7f39bc0a9480 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39c17b6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39bc0cf1e0 con 0x7f39a8004d50 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 0x7f39bc0aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 0x7f39bc0aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43014/0 (socket says 172.21.15.129:43014) 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.992+0000 7f39bb7fe640 1 -- 172.21.15.129:0/777165559 learned_addr learned my addr 172.21.15.129:0/777165559 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39baffd640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a8004d50 0x7f39bc154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b3fff640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0a9480 0x7f39bc144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39bb7fe640 1 -- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a8004d50 msgr2=0x7f39bc154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a8004d50 0x7f39bc154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39bb7fe640 1 -- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0a9480 msgr2=0x7f39bc144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0a9480 0x7f39bc144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39bb7fe640 1 -- 172.21.15.129:0/777165559 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0b3550 con 0x7f39bc0aab00 2024-04-25T00:43:08.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39bb7fe640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 0x7f39bc0aaf90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b0040 tx=0x7f39b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40b2040 con 0x7f39bc0aab00 2024-04-25T00:43:08.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b40a5ad0 con 0x7f39bc0aab00 2024-04-25T00:43:08.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4009c10 con 0x7f39bc0aab00 2024-04-25T00:43:08.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc0a9c60 con 0x7f39bc0aab00 2024-04-25T00:43:08.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc0aa220 con 0x7f39bc0aab00 2024-04-25T00:43:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc062e50 con 0x7f39bc0aab00 2024-04-25T00:43:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f39b40bf060 con 0x7f39bc0aab00 2024-04-25T00:43:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b37fe640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f399c0678c0 0x7f399c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39baffd640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f399c0678c0 0x7f399c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f39b411f6d0 con 0x7f39bc0aab00 2024-04-25T00:43:08.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:07.996+0000 7f39baffd640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f399c0678c0 0x7f399c069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39bc1f56b0 tx=0x7f39ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.000+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f39b40169d0 con 0x7f39bc0aab00 2024-04-25T00:43:08.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.160+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39bc075320 con 0x7f39bc0aab00 2024-04-25T00:43:08.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.160+0000 7f39b37fe640 1 -- 172.21.15.129:0/777165559 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f39b40f02a0 con 0x7f39bc0aab00 2024-04-25T00:43:08.166 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:43:08.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f399c0678c0 msgr2=0x7f399c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f399c0678c0 0x7f399c069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39bc1f56b0 tx=0x7f39ac008040 comp rx=0 tx=0).stop 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 msgr2=0x7f39bc0aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 0x7f39bc0aaf90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b0040 tx=0x7f39b4009750 comp rx=0 tx=0).stop 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 shutdown_connections 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f399c0678c0 0x7f399c069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39bc0aab00 0x7f39bc0aaf90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0a9480 0x7f39bc144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 --2- 172.21.15.129:0/777165559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39a8004d50 0x7f39bc154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 >> 172.21.15.129:0/777165559 conn(0x7f39bc067730 msgr2=0x7f39bc0ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 shutdown_connections 2024-04-25T00:43:08.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.164+0000 7f39c17b6640 1 -- 172.21.15.129:0/777165559 wait complete. 2024-04-25T00:43:08.185 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:43:08.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.296+0000 7ff421576640 1 Processor -- start 2024-04-25T00:43:08.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.296+0000 7ff421576640 1 -- start start 2024-04-25T00:43:08.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff41c15c590 0x7ff41c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41c06dbf0 con 0x7ff41c145960 2024-04-25T00:43:08.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff41c06cfe0 con 0x7ff41c15c590 2024-04-25T00:43:08.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c06d2c0 con 0x7ff41c0cccd0 2024-04-25T00:43:08.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff41c15c590 0x7ff41c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff41c15c590 0x7ff41c0cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43020/0 (socket says 172.21.15.129:43020) 2024-04-25T00:43:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 -- 172.21.15.129:0/2807713267 learned_addr learned my addr 172.21.15.129:0/2807713267 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 -- 172.21.15.129:0/2807713267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 msgr2=0x7ff41c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 --2- 172.21.15.129:0/2807713267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 -- 172.21.15.129:0/2807713267 >> v1:172.21.15.196:6789/0 conn(0x7ff41c145960 legacy=0x7ff41c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 -- 172.21.15.129:0/2807713267 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c06d5a0 con 0x7ff41c15c590 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff4127fc640 1 --2- 172.21.15.129:0/2807713267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c0cc790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff404009870 tx=0x7ff40401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82d8a77ab088add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff41a7fc640 1 -- 172.21.15.129:0/2807713267 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040b2070 con 0x7ff41c15c590 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff41a7fc640 1 -- 172.21.15.129:0/2807713267 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4040a9070 con 0x7ff41c15c590 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff41a7fc640 1 -- 172.21.15.129:0/2807713267 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040ad880 con 0x7ff41c15c590 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- 172.21.15.129:0/2807713267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 msgr2=0x7ff41c0cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 --2- 172.21.15.129:0/2807713267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c0cc790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff404009870 tx=0x7ff40401c490 comp rx=0 tx=0).stop 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- 172.21.15.129:0/2807713267 shutdown_connections 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 --2- 172.21.15.129:0/2807713267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 --2- 172.21.15.129:0/2807713267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c0cc790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- 172.21.15.129:0/2807713267 >> 172.21.15.129:0/2807713267 conn(0x7ff41c067730 msgr2=0x7ff41c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- 172.21.15.129:0/2807713267 shutdown_connections 2024-04-25T00:43:08.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.300+0000 7ff421576640 1 -- 172.21.15.129:0/2807713267 wait complete. 2024-04-25T00:43:08.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 Processor -- start 2024-04-25T00:43:08.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- start start 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff41c145960 0x7ff41c157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c136840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c145ec0 con 0x7ff41c15c590 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c145d40 con 0x7ff41c145960 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c06edd0 con 0x7ff41c0cccd0 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43028/0 (socket says 172.21.15.129:43028) 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 -- 172.21.15.129:0/974027928 learned_addr learned my addr 172.21.15.129:0/974027928 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41affd640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 -- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 msgr2=0x7ff41c156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff4127fc640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff41c145960 0x7ff41c157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 -- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff41c145960 msgr2=0x7ff41c157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff41c145960 0x7ff41c157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 -- 172.21.15.129:0/974027928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c152a10 con 0x7ff41c15c590 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41affd640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff41b7fe640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c136840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b970 tx=0x7ff41400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414019070 con 0x7ff41c15c590 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41400cb60 con 0x7ff41c15c590 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c152cf0 con 0x7ff41c15c590 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414014440 con 0x7ff41c15c590 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1532b0 con 0x7ff41c15c590 2024-04-25T00:43:08.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c062e50 con 0x7ff41c15c590 2024-04-25T00:43:08.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.304+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff41401c070 con 0x7ff41c15c590 2024-04-25T00:43:08.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.308+0000 7ff413fff640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff3fc0678c0 0x7ff3fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.308+0000 7ff41affd640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff3fc0678c0 0x7ff3fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.308+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff414085d50 con 0x7ff41c15c590 2024-04-25T00:43:08.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.308+0000 7ff41affd640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff3fc0678c0 0x7ff3fc069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0016e0 tx=0x7ff40c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.308+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff414023020 con 0x7ff41c15c590 2024-04-25T00:43:08.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.456+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff41c166cf0 con 0x7ff41c15c590 2024-04-25T00:43:08.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.456+0000 7ff413fff640 1 -- 172.21.15.129:0/974027928 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff414056a00 con 0x7ff41c15c590 2024-04-25T00:43:08.462 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:43:08.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff3fc0678c0 msgr2=0x7ff3fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff3fc0678c0 0x7ff3fc069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0016e0 tx=0x7ff40c009040 comp rx=0 tx=0).stop 2024-04-25T00:43:08.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 msgr2=0x7ff41c136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c136840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b970 tx=0x7ff41400be30 comp rx=0 tx=0).stop 2024-04-25T00:43:08.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 shutdown_connections 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff3fc0678c0 0x7ff3fc069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c15c590 0x7ff41c136840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff41c145960 0x7ff41c157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 --2- 172.21.15.129:0/974027928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff41c0cccd0 0x7ff41c156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 >> 172.21.15.129:0/974027928 conn(0x7ff41c067730 msgr2=0x7ff41c13ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 shutdown_connections 2024-04-25T00:43:08.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.460+0000 7ff421576640 1 -- 172.21.15.129:0/974027928 wait complete. 2024-04-25T00:43:08.479 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:43:08.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.584+0000 7fd830794640 1 Processor -- start 2024-04-25T00:43:08.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.584+0000 7fd830794640 1 -- start start 2024-04-25T00:43:08.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd82815f3b0 0x7fd82815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8280cc790 0x7fd8280ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cdf50 0x7fd8280ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd828160aa0 con 0x7fd82815f3b0 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd82815ff20 con 0x7fd8280cc790 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd828160200 con 0x7fd8280cdf50 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd82815f3b0 0x7fd82815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd82815f3b0 0x7fd82815ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43042/0 (socket says 172.21.15.129:43042) 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 -- 172.21.15.129:0/3339207956 learned_addr learned my addr 172.21.15.129:0/3339207956 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 -- 172.21.15.129:0/3339207956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cdf50 msgr2=0x7fd8280ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 --2- 172.21.15.129:0/3339207956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cdf50 0x7fd8280ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 -- 172.21.15.129:0/3339207956 >> v2:172.21.15.196:3300/0 conn(0x7fd8280cc790 msgr2=0x7fd8280ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 --2- 172.21.15.129:0/3339207956 >> v2:172.21.15.196:3300/0 conn(0x7fd8280cc790 0x7fd8280ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 -- 172.21.15.129:0/3339207956 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8281604e0 con 0x7fd82815f3b0 2024-04-25T00:43:08.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82e530640 1 --2- 172.21.15.129:0/3339207956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd82815f3b0 0x7fd82815ce70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8280b2410 tx=0x7fd82001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6803e0a60d1594ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82d52e640 1 -- 172.21.15.129:0/3339207956 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200b0070 con 0x7fd82815f3b0 2024-04-25T00:43:08.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82d52e640 1 -- 172.21.15.129:0/3339207956 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200ab600 con 0x7fd82815f3b0 2024-04-25T00:43:08.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd82d52e640 1 -- 172.21.15.129:0/3339207956 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200ab9d0 con 0x7fd82815f3b0 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- 172.21.15.129:0/3339207956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd82815f3b0 msgr2=0x7fd82815ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- 172.21.15.129:0/3339207956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd82815f3b0 0x7fd82815ce70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8280b2410 tx=0x7fd82001c490 comp rx=0 tx=0).stop 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- 172.21.15.129:0/3339207956 shutdown_connections 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- 172.21.15.129:0/3339207956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cdf50 0x7fd8280ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- 172.21.15.129:0/3339207956 >> v2:172.21.15.196:3300/0 conn(0x7fd8280cc790 0x7fd8280ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 --2- 172.21.15.129:0/3339207956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd82815f3b0 0x7fd82815ce70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- 172.21.15.129:0/3339207956 >> 172.21.15.129:0/3339207956 conn(0x7fd828067730 msgr2=0x7fd8280b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- 172.21.15.129:0/3339207956 shutdown_connections 2024-04-25T00:43:08.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.588+0000 7fd830794640 1 -- 172.21.15.129:0/3339207956 wait complete. 2024-04-25T00:43:08.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 Processor -- start 2024-04-25T00:43:08.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- start start 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cc790 0x7fd828148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 0x7fd82814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd82815f3b0 0x7fd828158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd82815f790 con 0x7fd8280cdf50 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd82815fba0 con 0x7fd82815f3b0 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd82815f910 con 0x7fd8280cc790 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 0x7fd82814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 0x7fd82814d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43054/0 (socket says 172.21.15.129:43054) 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 -- 172.21.15.129:0/3015313987 learned_addr learned my addr 172.21.15.129:0/3015313987 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82ed31640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd82815f3b0 0x7fd828158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82e530640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cc790 0x7fd828148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 -- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cc790 msgr2=0x7fd828148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cc790 0x7fd828148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 -- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd82815f3b0 msgr2=0x7fd828158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd82815f3b0 0x7fd828158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 -- 172.21.15.129:0/3015313987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82814e2e0 con 0x7fd8280cdf50 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd82dd2f640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 0x7fd82814d890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b410 tx=0x7fd81c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c015020 con 0x7fd8280cdf50 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81c011830 con 0x7fd8280cdf50 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82814e5c0 con 0x7fd8280cdf50 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c011c20 con 0x7fd8280cdf50 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd828148f60 con 0x7fd8280cdf50 2024-04-25T00:43:08.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.592+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd828062e50 con 0x7fd8280cdf50 2024-04-25T00:43:08.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.596+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd81c018020 con 0x7fd8280cdf50 2024-04-25T00:43:08.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.596+0000 7fd826ffd640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd8140678c0 0x7fd814069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.596+0000 7fd82e530640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd8140678c0 0x7fd814069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.596+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd81c085b60 con 0x7fd8280cdf50 2024-04-25T00:43:08.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.596+0000 7fd82e530640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd8140678c0 0x7fd814069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8200ad000 tx=0x7fd820008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.600+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd81c01f020 con 0x7fd8280cdf50 2024-04-25T00:43:08.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.756+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8280cc1e0 con 0x7fd8280cdf50 2024-04-25T00:43:08.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.756+0000 7fd826ffd640 1 -- 172.21.15.129:0/3015313987 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd81c0172c0 con 0x7fd8280cdf50 2024-04-25T00:43:08.763 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:43:08.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd8140678c0 msgr2=0x7fd814069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd8140678c0 0x7fd814069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8200ad000 tx=0x7fd820008040 comp rx=0 tx=0).stop 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 msgr2=0x7fd82814d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 0x7fd82814d890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b410 tx=0x7fd81c00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 shutdown_connections 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd8140678c0 0x7fd814069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd82815f3b0 0x7fd828158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8280cdf50 0x7fd82814d890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 --2- 172.21.15.129:0/3015313987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280cc790 0x7fd828148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 >> 172.21.15.129:0/3015313987 conn(0x7fd828067730 msgr2=0x7fd8280b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 shutdown_connections 2024-04-25T00:43:08.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.760+0000 7fd830794640 1 -- 172.21.15.129:0/3015313987 wait complete. 2024-04-25T00:43:08.779 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:43:08.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.880+0000 7f4ae370a640 1 Processor -- start 2024-04-25T00:43:08.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.880+0000 7f4ae370a640 1 -- start start 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae370a640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f4adc15bf50 0x7f4adc15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae370a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4adc0aa0c0 0x7f4adc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae370a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4adc06d1b0 con 0x7f4adc0aa550 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae370a640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f4adc06d490 con 0x7f4adc15bf50 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae370a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4adc06d770 con 0x7f4adc0aa0c0 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f4adc15bf50 0x7f4adc15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f4adc15bf50 0x7f4adc15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43068/0 (socket says 172.21.15.129:43068) 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 -- 172.21.15.129:0/3590929286 learned_addr learned my addr 172.21.15.129:0/3590929286 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae0ca5640 1 --2- 172.21.15.129:0/3590929286 >> v2:172.21.15.196:3300/0 conn(0x7f4adc0aa0c0 0x7f4adc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 -- 172.21.15.129:0/3590929286 >> v2:172.21.15.196:3300/0 conn(0x7f4adc0aa0c0 msgr2=0x7f4adc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:08.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 --2- 172.21.15.129:0/3590929286 >> v2:172.21.15.196:3300/0 conn(0x7f4adc0aa0c0 0x7f4adc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 -- 172.21.15.129:0/3590929286 >> v1:172.21.15.196:6789/0 conn(0x7f4adc0aa550 legacy=0x7f4adc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:08.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae14a6640 1 -- 172.21.15.129:0/3590929286 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc06da50 con 0x7f4adc15bf50 2024-04-25T00:43:08.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.884+0000 7f4ae0ca5640 1 --2- 172.21.15.129:0/3590929286 >> v2:172.21.15.196:3300/0 conn(0x7f4adc0aa0c0 0x7f4adc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:08.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae14a6640 1 --2- 172.21.15.129:0/3590929286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc15c330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0b3ae0 tx=0x7f4ad801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a327567c4cfa34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ad3fff640 1 -- 172.21.15.129:0/3590929286 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80b1070 con 0x7f4adc15bf50 2024-04-25T00:43:08.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ad3fff640 1 -- 172.21.15.129:0/3590929286 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad80ac620 con 0x7f4adc15bf50 2024-04-25T00:43:08.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ad3fff640 1 -- 172.21.15.129:0/3590929286 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80ac9f0 con 0x7f4adc15bf50 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/3590929286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 msgr2=0x7f4adc15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 --2- 172.21.15.129:0/3590929286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc15c330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0b3ae0 tx=0x7f4ad801c4b0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/3590929286 shutdown_connections 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 --2- 172.21.15.129:0/3590929286 >> v2:172.21.15.196:3300/0 conn(0x7f4adc0aa0c0 0x7f4adc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 --2- 172.21.15.129:0/3590929286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc15c330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/3590929286 >> 172.21.15.129:0/3590929286 conn(0x7f4adc0b14a0 msgr2=0x7f4adc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/3590929286 shutdown_connections 2024-04-25T00:43:08.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/3590929286 wait complete. 2024-04-25T00:43:08.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 Processor -- start 2024-04-25T00:43:08.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- start start 2024-04-25T00:43:08.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4adc0aa0c0 0x7f4adc148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4adc0aa550 0x7f4adc140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1691b0 con 0x7f4adc15bf50 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc169030 con 0x7f4adc0aa0c0 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4adc169330 con 0x7f4adc0aa550 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43070/0 (socket says 172.21.15.129:43070) 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 -- 172.21.15.129:0/129593671 learned_addr learned my addr 172.21.15.129:0/129593671 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae14a6640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4adc0aa0c0 0x7f4adc148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 -- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4adc0aa550 msgr2=0x7f4adc140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4adc0aa550 0x7f4adc140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 -- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4adc0aa0c0 msgr2=0x7f4adc148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4adc0aa0c0 0x7f4adc148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 -- 172.21.15.129:0/129593671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc142380 con 0x7f4adc15bf50 2024-04-25T00:43:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae1ca7640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc151420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c7b0 tx=0x7f4ad400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004b70 con 0x7f4adc15bf50 2024-04-25T00:43:08.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad4004d10 con 0x7f4adc15bf50 2024-04-25T00:43:08.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4adc147f30 con 0x7f4adc15bf50 2024-04-25T00:43:08.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400aa90 con 0x7f4adc15bf50 2024-04-25T00:43:08.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.888+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4adc1484f0 con 0x7f4adc15bf50 2024-04-25T00:43:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.892+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4adc062e50 con 0x7f4adc15bf50 2024-04-25T00:43:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.892+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ad400ac30 con 0x7f4adc15bf50 2024-04-25T00:43:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.892+0000 7f4ad1ffb640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4ac00678c0 0x7f4ac0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.892+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad4085f40 con 0x7f4adc15bf50 2024-04-25T00:43:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.892+0000 7f4ae14a6640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4ac00678c0 0x7f4ac0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:08.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.892+0000 7f4ae14a6640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4ac00678c0 0x7f4ac0069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80ae000 tx=0x7f4ad8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:08.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:08.896+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ad4020070 con 0x7f4adc15bf50 2024-04-25T00:43:09.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.076+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4adc075320 con 0x7f4adc15bf50 2024-04-25T00:43:09.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.076+0000 7f4ad1ffb640 1 -- 172.21.15.129:0/129593671 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f4adc142380 con 0x7f4adc15bf50 2024-04-25T00:43:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:09.082 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:42:58.001381+0000","modified":"2024-04-25T00:43:01.006515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-25T00:43:09.083 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T00:43:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4ac00678c0 msgr2=0x7f4ac0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4ac00678c0 0x7f4ac0069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80ae000 tx=0x7f4ad8008040 comp rx=0 tx=0).stop 2024-04-25T00:43:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 msgr2=0x7f4adc151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc151420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c7b0 tx=0x7f4ad400cc70 comp rx=0 tx=0).stop 2024-04-25T00:43:09.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 shutdown_connections 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4ac00678c0 0x7f4ac0069d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4adc0aa550 0x7f4adc140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4adc15bf50 0x7f4adc151420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 --2- 172.21.15.129:0/129593671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4adc0aa0c0 0x7f4adc148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 >> 172.21.15.129:0/129593671 conn(0x7f4adc0b14a0 msgr2=0x7f4adc15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 shutdown_connections 2024-04-25T00:43:09.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.080+0000 7f4ae370a640 1 -- 172.21.15.129:0/129593671 wait complete. 2024-04-25T00:43:09.100 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:43:09.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.200+0000 7f2102972640 1 Processor -- start 2024-04-25T00:43:09.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2102972640 1 -- start start 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2102972640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc15bf50 0x7f20fc15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2102972640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f20fc0aa0c0 0x7f20fc0aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2102972640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20fc06d1b0 con 0x7f20fc0ccb50 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2102972640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f20fc06d490 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2102972640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20fc06d770 con 0x7f20fc15bf50 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fb7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f20fc0aa0c0 0x7f20fc0aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2100f0f640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f20fc0ccb50 0x7f20fc06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46776/0 (socket says 172.21.15.129:46776) 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f2100f0f640 1 -- 172.21.15.129:0/2832815398 learned_addr learned my addr 172.21.15.129:0/2832815398 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fb7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f20fc0aa0c0 0x7f20fc0aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43080/0 (socket says 172.21.15.129:43080) 2024-04-25T00:43:09.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fbfff640 1 --2- 172.21.15.129:0/2832815398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc15bf50 0x7f20fc15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fb7fe640 1 -- 172.21.15.129:0/2832815398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc15bf50 msgr2=0x7f20fc15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:09.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fb7fe640 1 --2- 172.21.15.129:0/2832815398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc15bf50 0x7f20fc15c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20faffd640 1 -- 172.21.15.129:0/2832815398 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716992658 0 0) 0x7f20fc06d1b0 con 0x7f20fc0ccb50 2024-04-25T00:43:09.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fb7fe640 1 -- 172.21.15.129:0/2832815398 >> v1:172.21.15.129:6789/0 conn(0x7f20fc0ccb50 legacy=0x7f20fc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.204+0000 7f20fb7fe640 1 -- 172.21.15.129:0/2832815398 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc06da50 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f20fb7fe640 1 --2- 172.21.15.129:0/2832815398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc0aa520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20f00097c0 tx=0x7f20f001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d013aea23827db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:09.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f20faffd640 1 -- 172.21.15.129:0/2832815398 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00aa5f0 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f20faffd640 1 -- 172.21.15.129:0/2832815398 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f00af070 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f20faffd640 1 -- 172.21.15.129:0/2832815398 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00aa9b0 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 -- 172.21.15.129:0/2832815398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 msgr2=0x7f20fc0aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 --2- 172.21.15.129:0/2832815398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc0aa520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20f00097c0 tx=0x7f20f001c470 comp rx=0 tx=0).stop 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 -- 172.21.15.129:0/2832815398 shutdown_connections 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 --2- 172.21.15.129:0/2832815398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc0aa520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 --2- 172.21.15.129:0/2832815398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc15bf50 0x7f20fc15c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 -- 172.21.15.129:0/2832815398 >> 172.21.15.129:0/2832815398 conn(0x7f20fc0b14a0 msgr2=0x7f20fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 -- 172.21.15.129:0/2832815398 shutdown_connections 2024-04-25T00:43:09.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 -- 172.21.15.129:0/2832815398 wait complete. 2024-04-25T00:43:09.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 Processor -- start 2024-04-25T00:43:09.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.208+0000 7f2102972640 1 -- start start 2024-04-25T00:43:09.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc0ccb50 0x7f20fc140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc15bf50 0x7f20fc142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc06cee0 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc06cd60 con 0x7f20fc15bf50 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20fc169030 con 0x7f20fc0ccb50 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc1457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43082/0 (socket says 172.21.15.129:43082) 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 -- 172.21.15.129:0/3624750522 learned_addr learned my addr 172.21.15.129:0/3624750522 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 -- 172.21.15.129:0/3624750522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc0ccb50 msgr2=0x7f20fc140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc0ccb50 0x7f20fc140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 -- 172.21.15.129:0/3624750522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc15bf50 msgr2=0x7f20fc142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc15bf50 0x7f20fc142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 -- 172.21.15.129:0/3624750522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc140e40 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fbfff640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc1457a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d5f0 tx=0x7f20ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0125f0 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec019070 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc145e80 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0129d0 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc141d30 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc062e50 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f20ec012b70 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20f8ff9640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f20e80678c0 0x7f20e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20ec086190 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fb7fe640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f20e80678c0 0x7f20e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.212+0000 7f20fb7fe640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f20e80678c0 0x7f20e8069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ae000 tx=0x7f20f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:09.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.216+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f20fc140e40 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.364+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20fc075320 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.368+0000 7f20f8ff9640 1 -- 172.21.15.129:0/3624750522 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16641 (secure 0 0 0) 0x7f20ec056cd0 con 0x7f20fc0aa0c0 2024-04-25T00:43:09.373 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:09.373 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":20,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:42:59.648406+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:43:09.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f20e80678c0 msgr2=0x7f20e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f20e80678c0 0x7f20e8069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ae000 tx=0x7f20f0008040 comp rx=0 tx=0).stop 2024-04-25T00:43:09.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 msgr2=0x7f20fc1457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc1457a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d5f0 tx=0x7f20ec00dab0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 shutdown_connections 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f20e80678c0 0x7f20e8069d80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20fc0ccb50 0x7f20fc140900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc15bf50 0x7f20fc142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 --2- 172.21.15.129:0/3624750522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20fc0aa0c0 0x7f20fc1457a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 >> 172.21.15.129:0/3624750522 conn(0x7f20fc0b14a0 msgr2=0x7f20fc15aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 shutdown_connections 2024-04-25T00:43:09.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.372+0000 7f2102972640 1 -- 172.21.15.129:0/3624750522 wait complete. 2024-04-25T00:43:09.391 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:43:09.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.496+0000 7f115bef3640 1 Processor -- start 2024-04-25T00:43:09.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.496+0000 7f115bef3640 1 -- start start 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f115415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f11540cd260 0x7f11540aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f115406d1b0 con 0x7f11540aa0c0 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f115406d490 con 0x7f11540cd260 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f115406d770 con 0x7f115415bf50 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115a490640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f11540cd260 0x7f11540aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115948e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f11540aa0c0 0x7f11540ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46780/0 (socket says 172.21.15.129:46780) 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115948e640 1 -- 172.21.15.129:0/1351125875 learned_addr learned my addr 172.21.15.129:0/1351125875 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f1159c8f640 1 --2- 172.21.15.129:0/1351125875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f115415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115a490640 1 -- 172.21.15.129:0/1351125875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 msgr2=0x7f115415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115a490640 1 --2- 172.21.15.129:0/1351125875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f115415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115a490640 1 -- 172.21.15.129:0/1351125875 >> v1:172.21.15.129:6789/0 conn(0x7f11540aa0c0 legacy=0x7f11540ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115a490640 1 -- 172.21.15.129:0/1351125875 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115406da50 con 0x7f11540cd260 2024-04-25T00:43:09.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f1158c8d640 1 -- 172.21.15.129:0/1351125875 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971591420 0 0) 0x7f115406d1b0 con 0x7f11540aa0c0 2024-04-25T00:43:09.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115a490640 1 --2- 172.21.15.129:0/1351125875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f11540aa520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114c009d90 tx=0x7f114c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd96956d1c01adf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:09.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f1158c8d640 1 -- 172.21.15.129:0/1351125875 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c0b2070 con 0x7f11540cd260 2024-04-25T00:43:09.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f1158c8d640 1 -- 172.21.15.129:0/1351125875 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114c0a5a20 con 0x7f11540cd260 2024-04-25T00:43:09.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f1158c8d640 1 -- 172.21.15.129:0/1351125875 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c0a5df0 con 0x7f11540cd260 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- 172.21.15.129:0/1351125875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 msgr2=0x7f11540aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 --2- 172.21.15.129:0/1351125875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f11540aa520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114c009d90 tx=0x7f114c01c940 comp rx=0 tx=0).stop 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- 172.21.15.129:0/1351125875 shutdown_connections 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 --2- 172.21.15.129:0/1351125875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f11540aa520 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 --2- 172.21.15.129:0/1351125875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f115415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- 172.21.15.129:0/1351125875 >> 172.21.15.129:0/1351125875 conn(0x7f11540b14a0 msgr2=0x7f11540b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- 172.21.15.129:0/1351125875 shutdown_connections 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.500+0000 7f115bef3640 1 -- 172.21.15.129:0/1351125875 wait complete. 2024-04-25T00:43:09.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 Processor -- start 2024-04-25T00:43:09.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- start start 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11540aa0c0 0x7f1154152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f1154145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f11540a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115406ea60 con 0x7f11540cd260 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f115406cf70 con 0x7f11540aa0c0 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f115406ebe0 con 0x7f115415bf50 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f1154145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f1154145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43104/0 (socket says 172.21.15.129:43104) 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 -- 172.21.15.129:0/3375391728 learned_addr learned my addr 172.21.15.129:0/3375391728 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f1159c8f640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11540aa0c0 0x7f1154152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115a490640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f11540a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 -- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 msgr2=0x7f11540a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f11540a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 -- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11540aa0c0 msgr2=0x7f1154152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11540aa0c0 0x7f1154152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 -- 172.21.15.129:0/3375391728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1154147ad0 con 0x7f11540cd260 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115948e640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f1154145ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f114400c4d0 tx=0x7f114400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144018070 con 0x7f11540cd260 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11440074e0 con 0x7f11540cd260 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1154152ff0 con 0x7f11540cd260 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144004a30 con 0x7f11540cd260 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1154153770 con 0x7f11540cd260 2024-04-25T00:43:09.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.504+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1154062e50 con 0x7f11540cd260 2024-04-25T00:43:09.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.508+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f114401b070 con 0x7f11540cd260 2024-04-25T00:43:09.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.508+0000 7f114a7fc640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f11380678c0 0x7f1138069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:09.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.508+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1144084fa0 con 0x7f11540cd260 2024-04-25T00:43:09.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.508+0000 7f1159c8f640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f11380678c0 0x7f1138069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:09.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.508+0000 7f1159c8f640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f11380678c0 0x7f1138069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f115000b930 tx=0x7f1150008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:09.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.512+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1144055b70 con 0x7f11540cd260 2024-04-25T00:43:09.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.688+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1154075320 con 0x7f11540cd260 2024-04-25T00:43:09.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.688+0000 7f114a7fc640 1 -- 172.21.15.129:0/3375391728 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f1144055470 con 0x7f11540cd260 2024-04-25T00:43:09.694 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:43:09.694 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:42:58.001381+0000","modified":"2024-04-25T00:43:01.006515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-25T00:43:09.694 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T00:43:09.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f11380678c0 msgr2=0x7f1138069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f11380678c0 0x7f1138069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f115000b930 tx=0x7f1150008040 comp rx=0 tx=0).stop 2024-04-25T00:43:09.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 msgr2=0x7f1154145ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:09.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f1154145ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f114400c4d0 tx=0x7f114400c990 comp rx=0 tx=0).stop 2024-04-25T00:43:09.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 shutdown_connections 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f11380678c0 0x7f1138069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f115415bf50 0x7f11540a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11540cd260 0x7f1154145ca0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 --2- 172.21.15.129:0/3375391728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11540aa0c0 0x7f1154152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 >> 172.21.15.129:0/3375391728 conn(0x7f11540b14a0 msgr2=0x7f115414bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 shutdown_connections 2024-04-25T00:43:09.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:09.692+0000 7f115bef3640 1 -- 172.21.15.129:0/3375391728 wait complete. 2024-04-25T00:43:10.114 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:42:58.001381+0000', 'modified': '2024-04-25T00:43:01.006515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6833/1985283630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 1985283630}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 1985283630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4287, 'qdb_cluster': [4287]} 2024-04-25T00:43:10.114 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:43:10.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:43:10.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 Processor -- start 2024-04-25T00:43:10.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 -- start start 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400aa0c0 0x7f04400aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f04400ccb50 0x7f04400cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044006d1b0 con 0x7f044015bf50 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f044015ca40 con 0x7f04400ccb50 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0447f98640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f044006d490 con 0x7f04400aa0c0 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0446535640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f04400ccb50 0x7f04400cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0445d34640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f044015bf50 0x7f044015c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46790/0 (socket says 172.21.15.129:46790) 2024-04-25T00:43:10.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0445d34640 1 -- 172.21.15.129:0/442309964 learned_addr learned my addr 172.21.15.129:0/442309964 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:10.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0445533640 1 --2- 172.21.15.129:0/442309964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400aa0c0 0x7f04400aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0446535640 1 -- 172.21.15.129:0/442309964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400aa0c0 msgr2=0x7f04400aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0446535640 1 --2- 172.21.15.129:0/442309964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400aa0c0 0x7f04400aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0446535640 1 -- 172.21.15.129:0/442309964 >> v1:172.21.15.129:6789/0 conn(0x7f044015bf50 legacy=0x7f044015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0444d32640 1 -- 172.21.15.129:0/442309964 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771846954 0 0) 0x7f044006d1b0 con 0x7f044015bf50 2024-04-25T00:43:10.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.240+0000 7f0446535640 1 -- 172.21.15.129:0/442309964 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044006d770 con 0x7f04400ccb50 2024-04-25T00:43:10.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0446535640 1 --2- 172.21.15.129:0/442309964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400ccb50 0x7f04400cd000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0430009d90 tx=0x7f043001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f541773b67d7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0444d32640 1 -- 172.21.15.129:0/442309964 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300b2070 con 0x7f04400ccb50 2024-04-25T00:43:10.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0444d32640 1 -- 172.21.15.129:0/442309964 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04300a5920 con 0x7f04400ccb50 2024-04-25T00:43:10.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0444d32640 1 -- 172.21.15.129:0/442309964 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300a5cf0 con 0x7f04400ccb50 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 -- 172.21.15.129:0/442309964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400ccb50 msgr2=0x7f04400cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 --2- 172.21.15.129:0/442309964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400ccb50 0x7f04400cd000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0430009d90 tx=0x7f043001c970 comp rx=0 tx=0).stop 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 -- 172.21.15.129:0/442309964 shutdown_connections 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 --2- 172.21.15.129:0/442309964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400ccb50 0x7f04400cd000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 --2- 172.21.15.129:0/442309964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400aa0c0 0x7f04400aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 -- 172.21.15.129:0/442309964 >> 172.21.15.129:0/442309964 conn(0x7f04400b14a0 msgr2=0x7f04400b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 -- 172.21.15.129:0/442309964 shutdown_connections 2024-04-25T00:43:10.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 -- 172.21.15.129:0/442309964 wait complete. 2024-04-25T00:43:10.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 Processor -- start 2024-04-25T00:43:10.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.244+0000 7f0447f98640 1 -- start start 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 0x7f04400b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400ccb50 0x7f044013b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f044015bf50 0x7f0440149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f044006ea60 con 0x7f04400aa0c0 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f044006cf70 con 0x7f044015bf50 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f044006ebe0 con 0x7f04400ccb50 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 0x7f04400b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 0x7f04400b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43130/0 (socket says 172.21.15.129:43130) 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 -- 172.21.15.129:0/172371939 learned_addr learned my addr 172.21.15.129:0/172371939 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:10.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445533640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400ccb50 0x7f044013b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 -- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400ccb50 msgr2=0x7f044013b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400ccb50 0x7f044013b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 -- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f044015bf50 msgr2=0x7f0440149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0446535640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f044015bf50 0x7f0440149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f044015bf50 0x7f0440149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 -- 172.21.15.129:0/172371939 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044014d960 con 0x7f04400aa0c0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445d34640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 0x7f04400b0e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f043800c4b0 tx=0x7f043800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f043e7fc640 1 -- 172.21.15.129:0/172371939 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438018070 con 0x7f04400aa0c0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f043e7fc640 1 -- 172.21.15.129:0/172371939 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0438007500 con 0x7f04400aa0c0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f043e7fc640 1 -- 172.21.15.129:0/172371939 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004a30 con 0x7f04400aa0c0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044014dc40 con 0x7f04400aa0c0 2024-04-25T00:43:10.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044014e200 con 0x7f04400aa0c0 2024-04-25T00:43:10.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f043e7fc640 1 -- 172.21.15.129:0/172371939 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0438004bd0 con 0x7f04400aa0c0 2024-04-25T00:43:10.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f043e7fc640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f042c0678c0 0x7f042c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f043e7fc640 1 -- 172.21.15.129:0/172371939 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0438084e20 con 0x7f04400aa0c0 2024-04-25T00:43:10.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445533640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f042c0678c0 0x7f042c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f0445533640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f042c0678c0 0x7f042c069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04400ae490 tx=0x7f0434008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.248+0000 7f044b111000 1 Processor -- start 2024-04-25T00:43:10.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- start start 2024-04-25T00:43:10.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b60fef0 0x55fb5b6102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b5c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb5b4145e0 con 0x55fb5b610810 2024-04-25T00:43:10.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb5b39bc50 con 0x55fb5b60fef0 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55fb5b4105a0 con 0x55fb5b5c6bc0 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43142/0 (socket says 172.21.15.129:43142) 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 -- 172.21.15.129:0/1348773985 learned_addr learned my addr 172.21.15.129:0/1348773985 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445d34640 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b60fef0 0x55fb5b6102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0446535640 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b5c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 -- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 msgr2=0x55fb5b5c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b5c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 -- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b60fef0 msgr2=0x55fb5b6102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b60fef0 0x55fb5b6102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 -- 172.21.15.129:0/1348773985 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5b399d10 con 0x55fb5b610810 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445d34640 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b60fef0 0x55fb5b6102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445533640 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c6680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f043400db60 tx=0x7f0434010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec94603c505adc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f04277fe640 1 -- 172.21.15.129:0/1348773985 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434019050 con 0x55fb5b610810 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f04277fe640 1 -- 172.21.15.129:0/1348773985 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434023070 con 0x55fb5b610810 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f04277fe640 1 -- 172.21.15.129:0/1348773985 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434012bb0 con 0x55fb5b610810 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- 172.21.15.129:0/1348773985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 msgr2=0x55fb5b5c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c6680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f043400db60 tx=0x7f0434010000 comp rx=0 tx=0).stop 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- 172.21.15.129:0/1348773985 shutdown_connections 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b5c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c6680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- 172.21.15.129:0/1348773985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b60fef0 0x55fb5b6102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- 172.21.15.129:0/1348773985 >> 172.21.15.129:0/1348773985 conn(0x55fb5b5c2b70 msgr2=0x55fb5b5c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- 172.21.15.129:0/1348773985 shutdown_connections 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- 172.21.15.129:0/1348773985 wait complete. 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- start start 2024-04-25T00:43:10.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 Processor -- start 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b5f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb5b3ce520 con 0x55fb5b610810 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb5b5c29f0 con 0x55fb5b5c6bc0 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55fb5b3acdd0 con 0x55fb5b60fef0 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f044b111000 10 client.0 _ll_register_callbacks cb 0x55fb5b5b15c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0446535640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0446535640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43156/0 (socket says 172.21.15.129:43156) 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0446535640 1 -- 172.21.15.129:0/365338682 learned_addr learned my addr 172.21.15.129:0/365338682 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.252+0000 7f0445d34640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b5f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0445533640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0446535640 1 -- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 msgr2=0x55fb5b5f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0446535640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0446535640 1 -- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 msgr2=0x55fb5b5f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0446535640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b5f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0446535640 1 -- 172.21.15.129:0/365338682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5b3ac7b0 con 0x55fb5b610810 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0446535640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5f59a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04300b0040 tx=0x7f043001b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db728f290e62ce70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 1 -- 172.21.15.129:0/365338682 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300be070 con 0x55fb5b610810 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 1 -- 172.21.15.129:0/365338682 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f04300b8450 con 0x55fb5b610810 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0434023070 con 0x55fb5b610810 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 1 -- 172.21.15.129:0/365338682 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f04300c8050 con 0x55fb5b610810 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 10 client.4629 fetch_fsmap learned FSMap version 7 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55fb5b39bea0 con 0x55fb5b610810 2024-04-25T00:43:10.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 1 -- 172.21.15.129:0/365338682 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434023070 con 0x55fb5b610810 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 1 -- 172.21.15.129:0/365338682 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f04300cb020 con 0x55fb5b610810 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 10 client.4629 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 10 client.4629 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 20 client.4629 populate_metadata read hostname 'smithi129' 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b5f6680 0x55fb5b621fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 4 client.4629 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 --> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5b3afab0 con 0x55fb5b5f6680 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0445d34640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b5f6680 0x55fb5b621fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 10 client.4629 ms_handle_connect on v2:172.21.15.196:6832/1985283630 2024-04-25T00:43:10.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0445d34640 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b5f6680 0x55fb5b621fa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9fadb1f00a7f61a server_cookie=821f6bbcb138eb01 in_seq=0 out_seq=0 2024-04-25T00:43:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 1 -- 172.21.15.129:0/365338682 <== mds.0 v2:172.21.15.196:6832/1985283630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb5b3afab0 con 0x55fb5b5f6680 2024-04-25T00:43:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 10 client.4629 handle_command_reply: tid=0 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 client.4629 shutdown 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 msgr2=0x55fb5b5f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5f59a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04300b0040 tx=0x7f043001b1b0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 shutdown_connections 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b5f6680 0x55fb5b621fa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5f59a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 --2- 172.21.15.129:0/365338682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b5f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f0424ff9640 0 client.4629 ms_handle_reset on v2:172.21.15.196:6832/1985283630 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 >> 172.21.15.129:0/365338682 conn(0x55fb5b5c2b70 msgr2=0x55fb5b5c57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 shutdown_connections 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.256+0000 7f044b111000 1 -- 172.21.15.129:0/365338682 wait complete. 2024-04-25T00:43:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.260+0000 7f044b111000 20 client.4629 trim_cache size 0 max 16384 2024-04-25T00:43:10.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 Processor -- start 2024-04-25T00:43:10.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- start start 2024-04-25T00:43:10.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b60f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04400cf330 con 0x55fb5b610810 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04400af900 con 0x55fb5b5c6bc0 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04400cece0 con 0x55fb5b60fef0 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43170/0 (socket says 172.21.15.129:43170) 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 -- 172.21.15.129:0/2172626693 learned_addr learned my addr 172.21.15.129:0/2172626693 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 -- 172.21.15.129:0/2172626693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 msgr2=0x55fb5b5c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 -- 172.21.15.129:0/2172626693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 msgr2=0x55fb5b60f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b60f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 -- 172.21.15.129:0/2172626693 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5b39ce80 con 0x55fb5b610810 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0446535640 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c64b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f043001c7a0 tx=0x7f04300b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd251e8d2b857d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0426ffd640 1 -- 172.21.15.129:0/2172626693 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300be040 con 0x55fb5b610810 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0426ffd640 1 -- 172.21.15.129:0/2172626693 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04300a5400 con 0x55fb5b610810 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f0426ffd640 1 -- 172.21.15.129:0/2172626693 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04300b2040 con 0x55fb5b610810 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- 172.21.15.129:0/2172626693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 msgr2=0x55fb5b5c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c64b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f043001c7a0 tx=0x7f04300b8000 comp rx=0 tx=0).stop 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- 172.21.15.129:0/2172626693 shutdown_connections 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b60fef0 0x55fb5b5c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b610810 0x55fb5b5c64b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 --2- 172.21.15.129:0/2172626693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b5c6bc0 0x55fb5b60f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- 172.21.15.129:0/2172626693 >> 172.21.15.129:0/2172626693 conn(0x55fb5b5c2b70 msgr2=0x55fb5b60eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- 172.21.15.129:0/2172626693 shutdown_connections 2024-04-25T00:43:10.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- 172.21.15.129:0/2172626693 wait complete. 2024-04-25T00:43:10.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 -- start start 2024-04-25T00:43:10.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.276+0000 7f044b111000 1 Processor -- start 2024-04-25T00:43:10.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b67c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 0x55fb5b622c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b610810 0x55fb5b623160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb5b3c8c40 con 0x55fb5b60fef0 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fb5b4c6e80 con 0x55fb5b610810 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55fb5b3dd810 con 0x55fb5b5c6bc0 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 10 client.0 _ll_register_callbacks cb 0x7f04300b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 0x55fb5b622c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 0x55fb5b622c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43184/0 (socket says 172.21.15.129:43184) 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 -- 172.21.15.129:0/2377218397 learned_addr learned my addr 172.21.15.129:0/2377218397 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0446535640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b610810 0x55fb5b623160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445d34640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b67c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 -- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 msgr2=0x55fb5b67c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b67c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 -- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b610810 msgr2=0x55fb5b623160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b610810 0x55fb5b623160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 -- 172.21.15.129:0/2377218397 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5b3e1760 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445533640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 0x55fb5b622c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f043400db00 tx=0x7f04340046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ae4ca2159bc3f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 1 -- 172.21.15.129:0/2377218397 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04340044f0 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04300a5400 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 1 -- 172.21.15.129:0/2377218397 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0434003620 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 1 -- 172.21.15.129:0/2377218397 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f0434019020 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 10 client.4635 fetch_fsmap learned FSMap version 7 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55fb5b46d9e0 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 1 -- 172.21.15.129:0/2377218397 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04300a5400 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 1 -- 172.21.15.129:0/2377218397 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f043402f020 con 0x55fb5b60fef0 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 10 client.4635 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 10 client.4635 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 20 client.4635 populate_metadata read hostname 'smithi129' 2024-04-25T00:43:10.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b6236a0 0x55fb5b623a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 4 client.4635 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 --> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] -- command(tid 0: {"prefix": "status"}) -- 0x55fb5b421b20 con 0x55fb5b6236a0 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445d34640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b6236a0 0x55fb5b623a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 10 client.4635 ms_handle_connect on v2:172.21.15.196:6832/1985283630 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0445d34640 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b6236a0 0x55fb5b623a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c3ace0ea0adcef server_cookie=11999f3332df7172 in_seq=0 out_seq=0 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 1 -- 172.21.15.129:0/2377218397 <== mds.0 v2:172.21.15.196:6832/1985283630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55fb5b421b20 con 0x55fb5b6236a0 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 10 client.4635 handle_command_reply: tid=0 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 client.4635 shutdown 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 msgr2=0x55fb5b622c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 0x55fb5b622c20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f043400db00 tx=0x7f04340046d0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 shutdown_connections 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:6832/1985283630,v1:172.21.15.196:6833/1985283630] conn(0x55fb5b6236a0 0x55fb5b623a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fb5b610810 0x55fb5b623160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb5b60fef0 0x55fb5b622c20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0424ff9640 0 client.4635 ms_handle_reset on v2:172.21.15.196:6832/1985283630 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 --2- 172.21.15.129:0/2377218397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fb5b5c6bc0 0x55fb5b67c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 >> 172.21.15.129:0/2377218397 conn(0x55fb5b5c2b70 msgr2=0x55fb5b60f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 shutdown_connections 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 1 -- 172.21.15.129:0/2377218397 wait complete. 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f044b111000 20 client.4635 trim_cache size 0 max 16384 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "aaa4363a-84af-4fd8-b222-fe97e0bf8f21", 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4287, 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-04-25T00:43:10.291 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 11.637393643999999, 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 7, 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 20, 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 18.134801542999998 2024-04-25T00:43:10.292 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f042c0678c0 msgr2=0x7f042c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f042c0678c0 0x7f042c069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04400ae490 tx=0x7f0434008040 comp rx=0 tx=0).stop 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 msgr2=0x7f04400b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 0x7f04400b0e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f043800c4b0 tx=0x7f043800c970 comp rx=0 tx=0).stop 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 shutdown_connections 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f042c0678c0 0x7f042c069d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04400ccb50 0x7f044013b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f044015bf50 0x7f0440149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 --2- 172.21.15.129:0/172371939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04400aa0c0 0x7f04400b0e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 >> 172.21.15.129:0/172371939 conn(0x7f04400b14a0 msgr2=0x7f044014ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 shutdown_connections 2024-04-25T00:43:10.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.280+0000 7f0447f98640 1 -- 172.21.15.129:0/172371939 wait complete. 2024-04-25T00:43:10.302 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:43:10.312 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:43:10.312 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:43:10.313 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:43:10.313 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T00:43:10.349 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T00:43:10.349 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T00:43:10.349 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T00:43:10.349 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:10.350 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:10.350 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:10.350 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:43:10.350 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:10.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:10.350 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T00:43:10.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:10.358 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-25T00:43:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-04-25T00:43:10.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:43:10.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:10.421 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T00:43:10.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:10.427 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-04-25T00:43:10.485 INFO:teuthology.orchestra.run.smithi129.stderr:Cannot find device "ceph-brx" 2024-04-25T00:43:10.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:43:10.487 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T00:43:10.487 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:43:10.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.648+0000 7fa1b1c99640 1 Processor -- start 2024-04-25T00:43:10.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.648+0000 7fa1b1c99640 1 -- start start 2024-04-25T00:43:10.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1b1c99640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa1ac15bf50 0x7fa1ac15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1b1c99640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ac06d1b0 con 0x7fa1ac0aa0c0 2024-04-25T00:43:10.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1b1c99640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ac06d490 con 0x7fa1ac0cd260 2024-04-25T00:43:10.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1b1c99640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ac06d770 con 0x7fa1ac15bf50 2024-04-25T00:43:10.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aaffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fa1ac0aa0c0 0x7fa1ac0ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46810/0 (socket says 172.21.15.129:46810) 2024-04-25T00:43:10.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aaffd640 1 -- 172.21.15.129:0/251879634 learned_addr learned my addr 172.21.15.129:0/251879634 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:10.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1ab7fe640 1 --2- 172.21.15.129:0/251879634 >> v2:172.21.15.129:3300/0 conn(0x7fa1ac15bf50 0x7fa1ac15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aa7fc640 1 -- 172.21.15.129:0/251879634 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1126226345 0 0) 0x7fa1ac06d1b0 con 0x7fa1ac0aa0c0 2024-04-25T00:43:10.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aa7fc640 1 -- 172.21.15.129:0/251879634 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa198003530 con 0x7fa1ac0aa0c0 2024-04-25T00:43:10.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1ab7fe640 1 -- 172.21.15.129:0/251879634 >> v1:172.21.15.196:6789/0 conn(0x7fa1ac0cd260 legacy=0x7fa1ac06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1ab7fe640 1 -- 172.21.15.129:0/251879634 >> v1:172.21.15.129:6789/0 conn(0x7fa1ac0aa0c0 legacy=0x7fa1ac0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1ab7fe640 1 -- 172.21.15.129:0/251879634 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac06da50 con 0x7fa1ac15bf50 2024-04-25T00:43:10.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1ab7fe640 1 --2- 172.21.15.129:0/251879634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac15bf50 0x7fa1ac15c330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0097c0 tx=0x7fa19c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc956928f826930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aa7fc640 1 -- 172.21.15.129:0/251879634 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0b2070 con 0x7fa1ac15bf50 2024-04-25T00:43:10.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aa7fc640 1 -- 172.21.15.129:0/251879634 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c0ad680 con 0x7fa1ac15bf50 2024-04-25T00:43:10.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.652+0000 7fa1aa7fc640 1 -- 172.21.15.129:0/251879634 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0ada30 con 0x7fa1ac15bf50 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- 172.21.15.129:0/251879634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac15bf50 msgr2=0x7fa1ac15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/251879634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac15bf50 0x7fa1ac15c330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0097c0 tx=0x7fa19c01c470 comp rx=0 tx=0).stop 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- 172.21.15.129:0/251879634 shutdown_connections 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/251879634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac15bf50 0x7fa1ac15c330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- 172.21.15.129:0/251879634 >> 172.21.15.129:0/251879634 conn(0x7fa1ac0b14a0 msgr2=0x7fa1ac0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- 172.21.15.129:0/251879634 shutdown_connections 2024-04-25T00:43:10.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- 172.21.15.129:0/251879634 wait complete. 2024-04-25T00:43:10.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 Processor -- start 2024-04-25T00:43:10.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- start start 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1ac0aa0c0 0x7fa1ac14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 0x7fa1ac0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1ac15bf50 0x7fa1ac13af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac16a1c0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac16a040 con 0x7fa1ac0aa0c0 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1b1c99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ac16a340 con 0x7fa1ac15bf50 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 0x7fa1ac0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 0x7fa1ac0cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43196/0 (socket says 172.21.15.129:43196) 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 -- 172.21.15.129:0/2534176429 learned_addr learned my addr 172.21.15.129:0/2534176429 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1ab7fe640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1ac0aa0c0 0x7fa1ac14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1abfff640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1ac15bf50 0x7fa1ac13af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 -- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1ac15bf50 msgr2=0x7fa1ac13af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1ac15bf50 0x7fa1ac13af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 -- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1ac0aa0c0 msgr2=0x7fa1ac14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1ac0aa0c0 0x7fa1ac14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 -- 172.21.15.129:0/2534176429 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac0cfcd0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1aaffd640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 0x7fa1ac0cf790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c500 tx=0x7fa19400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194017070 con 0x7fa1ac0cd260 2024-04-25T00:43:10.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.656+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa194005cf0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac1495f0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940072f0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac0cf270 con 0x7fa1ac0cd260 2024-04-25T00:43:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac1499e0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa19401a070 con 0x7fa1ac0cd260 2024-04-25T00:43:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1a3fff640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa18c0678c0 0x7fa18c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa194084fb0 con 0x7fa1ac0cd260 2024-04-25T00:43:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1ab7fe640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa18c0678c0 0x7fa18c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.660+0000 7fa1ab7fe640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa18c0678c0 0x7fa18c069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0af000 tx=0x7fa19c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:10.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.664+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa194055530 con 0x7fa1ac0cd260 2024-04-25T00:43:10.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1ac062e50 con 0x7fa1ac0cd260 2024-04-25T00:43:10.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1a3fff640 1 -- 172.21.15.129:0/2534176429 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fa194059830 con 0x7fa1ac0cd260 2024-04-25T00:43:10.864 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:43:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa18c0678c0 msgr2=0x7fa18c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa18c0678c0 0x7fa18c069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0af000 tx=0x7fa19c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 msgr2=0x7fa1ac0cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 0x7fa1ac0cf790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c500 tx=0x7fa19400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 shutdown_connections 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa18c0678c0 0x7fa18c069d80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1ac15bf50 0x7fa1ac13af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1ac0cd260 0x7fa1ac0cf790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 --2- 172.21.15.129:0/2534176429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1ac0aa0c0 0x7fa1ac14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 >> 172.21.15.129:0/2534176429 conn(0x7fa1ac0b14a0 msgr2=0x7fa1ac0a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 shutdown_connections 2024-04-25T00:43:10.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10.860+0000 7fa1b1c99640 1 -- 172.21.15.129:0/2534176429 wait complete. 2024-04-25T00:43:10.881 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:43:10.881 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:43:10.881 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-04-25T00:43:10.882 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:43:10.882 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:10.882 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:43:10.882 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:43:10.882 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:43:10.882 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:43:10.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:10.882 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:10.888 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42478sec preferred_lft 42478sec 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::ec4:7aff:fe88:7261/64 scope link 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 02:42:b7:0a:c9:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:10.889 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:43:10.890 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:10.890 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:43:10.890 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:10.890 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:10.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:43:10.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-04-25T00:43:10.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-04-25T00:43:10.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:43:10.891 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:10.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:11.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:11.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:11.010 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:43:11.062 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-04-25T00:43:11.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:11.063 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:11.130 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:43:11.131 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:11.131 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:11.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:43:11.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:43:11.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:43:11.131 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:11.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:11.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:11.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:11.242 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T00:43:11.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:11.287 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-04-25T00:43:11.335 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:11.335 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:11.335 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:11.335 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:43:11.335 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:11.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:11.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:11.448 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:43:11.449 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:11.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:11.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:11.588 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:11.588 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:11.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-04-25T00:43:11.588 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:43:11.588 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:11.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:11.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:11.690 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:43:11.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:11.690 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:11.739 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:43:11.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:11.739 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:11.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T00:43:11.840 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:11.887 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/run/lock 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T00:43:11.888 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/fs/nfsd 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/run/rpc_pipefs 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:43:11.889 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:43:11.890 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:43:11.890 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T00:43:11.890 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:11.890 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:11.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:11.891 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T00:43:11.938 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:43:11.939 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:43:11.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T00:43:11.997 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T00:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T00:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T00:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/run/lock 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T00:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/fs/nfsd 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/run/rpc_pipefs 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T00:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:12.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:12.047 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T00:43:12.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:2024-04-25T00:43:12.056+0000 7fdac340a580 -1 init, newargv = 0x55a89f5b5980 newargc=15 2024-04-25T00:43:12.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[34950]: starting ceph client 2024-04-25T00:43:12.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[34950]: starting fuse 2024-04-25T00:43:13.165 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T00:43:13.181 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T00:43:13.228 INFO:teuthology.orchestra.run.smithi129.stdout:/run/lock 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/fs/nfsd 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/run/rpc_pipefs 2024-04-25T00:43:13.229 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:13.230 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:13.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:13.231 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T00:43:13.283 INFO:teuthology.orchestra.run.smithi129.stdout:45 2024-04-25T00:43:13.283 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:43:13.283 DEBUG:teuthology.orchestra.run.smithi129:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:43:13.283 DEBUG:teuthology.orchestra.run.smithi129:> import glob 2024-04-25T00:43:13.283 DEBUG:teuthology.orchestra.run.smithi129:> import re 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> import os 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> import subprocess 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> def _find_admin_socket(client_name): 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> files = glob.glob(asok_path) 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> # Given a non-glob path, it better be there 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> if "*" not in asok_path: 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> assert(len(files) == 1) 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> return files[0] 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> for f in files: 2024-04-25T00:43:13.284 DEBUG:teuthology.orchestra.run.smithi129:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> contents = proc_f.read() 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> if mountpoint in contents: 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> return f 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> print(_find_admin_socket("client.0")) 2024-04-25T00:43:13.285 DEBUG:teuthology.orchestra.run.smithi129:> ' 2024-04-25T00:43:13.408 INFO:teuthology.orchestra.run.smithi129.stdout:/var/run/ceph/ceph-client.0.34950.asok 2024-04-25T00:43:13.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:13.419 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34950.asok 2024-04-25T00:43:13.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:13.420 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34950.asok status 2024-04-25T00:43:13.572 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "metadata": { 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "entity_id": "0", 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": "smithi129", 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "pid": "34950", 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "root": "/" 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_count": 0, 2024-04-25T00:43:13.573 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_pinned_count": 0, 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4653, 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "inst": { 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "name": { 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "client", 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "num": 4653 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2501790063 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T00:43:13.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2501790063 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "inst_str": "client.4653 192.168.0.1:0/2501790063", 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "addr_str": "192.168.0.1:0/2501790063", 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "inode_count": 1, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_epoch": 7, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch": 20, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklisted": false, 2024-04-25T00:43:13.575 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs" 2024-04-25T00:43:13.576 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:43:13.587 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:43:13.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 Processor -- start 2024-04-25T00:43:13.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 -- start start 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f57d015bf50 0x7f57d015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00aa0c0 0x7f57d00aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f57d00ccb50 0x7f57d00cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f57d006d1b0 con 0x7f57d015bf50 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f57d006d490 con 0x7f57d00ccb50 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d7da8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f57d006d770 con 0x7f57d00aa0c0 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f57d015bf50 0x7f57d015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f57d015bf50 0x7f57d015c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55920/0 (socket says 172.21.15.129:55920) 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 -- 172.21.15.129:0/2605899995 learned_addr learned my addr 172.21.15.129:0/2605899995 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d6345640 1 --2- 172.21.15.129:0/2605899995 >> v2:172.21.15.196:3300/0 conn(0x7f57d00ccb50 0x7f57d00cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5343640 1 --2- 172.21.15.129:0/2605899995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00aa0c0 0x7f57d00aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 -- 172.21.15.129:0/2605899995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00aa0c0 msgr2=0x7f57d00aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 --2- 172.21.15.129:0/2605899995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00aa0c0 0x7f57d00aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 -- 172.21.15.129:0/2605899995 >> v2:172.21.15.196:3300/0 conn(0x7f57d00ccb50 msgr2=0x7f57d00cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 --2- 172.21.15.129:0/2605899995 >> v2:172.21.15.196:3300/0 conn(0x7f57d00ccb50 0x7f57d00cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 -- 172.21.15.129:0/2605899995 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d006da50 con 0x7f57d015bf50 2024-04-25T00:43:13.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d5b44640 1 --2- 172.21.15.129:0/2605899995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d015bf50 0x7f57d015c330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f57d0072d80 tx=0x7f57cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825397bff2d32d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:13.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d4b42640 1 -- 172.21.15.129:0/2605899995 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0ab5f0 con 0x7f57d015bf50 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d4b42640 1 -- 172.21.15.129:0/2605899995 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc0b0070 con 0x7f57d015bf50 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.708+0000 7f57d4b42640 1 -- 172.21.15.129:0/2605899995 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0ab9b0 con 0x7f57d015bf50 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/2605899995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d015bf50 msgr2=0x7f57d015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- 172.21.15.129:0/2605899995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d015bf50 0x7f57d015c330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f57d0072d80 tx=0x7f57cc01c520 comp rx=0 tx=0).stop 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/2605899995 shutdown_connections 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- 172.21.15.129:0/2605899995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00aa0c0 0x7f57d00aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- 172.21.15.129:0/2605899995 >> v2:172.21.15.196:3300/0 conn(0x7f57d00ccb50 0x7f57d00cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- 172.21.15.129:0/2605899995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d015bf50 0x7f57d015c330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/2605899995 >> 172.21.15.129:0/2605899995 conn(0x7f57d00b14a0 msgr2=0x7f57d00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/2605899995 shutdown_connections 2024-04-25T00:43:13.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/2605899995 wait complete. 2024-04-25T00:43:13.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 Processor -- start 2024-04-25T00:43:13.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- start start 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 0x7f57d0141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00ccb50 0x7f57d0146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57d015bf50 0x7f57d01469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57d006ea60 con 0x7f57d00aa0c0 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57d006cf70 con 0x7f57d015bf50 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f57d006ebe0 con 0x7f57d00ccb50 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 0x7f57d0141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 0x7f57d0141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55928/0 (socket says 172.21.15.129:55928) 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 -- 172.21.15.129:0/3857693556 learned_addr learned my addr 172.21.15.129:0/3857693556 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5343640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00ccb50 0x7f57d0146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d6345640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57d015bf50 0x7f57d01469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 -- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00ccb50 msgr2=0x7f57d0146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00ccb50 0x7f57d0146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 -- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57d015bf50 msgr2=0x7f57d01469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57d015bf50 0x7f57d01469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 -- 172.21.15.129:0/3857693556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d004fdf0 con 0x7f57d00aa0c0 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5343640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00ccb50 0x7f57d0146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d6345640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57d015bf50 0x7f57d01469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d5b44640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 0x7f57d0141a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57cc01c350 tx=0x7f57cc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0abe60 con 0x7f57d00aa0c0 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc0be5f0 con 0x7f57d00aa0c0 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0be9a0 con 0x7f57d00aa0c0 2024-04-25T00:43:13.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d01435c0 con 0x7f57d00aa0c0 2024-04-25T00:43:13.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.712+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d0143e60 con 0x7f57d00aa0c0 2024-04-25T00:43:13.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.716+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d0062e50 con 0x7f57d00aa0c0 2024-04-25T00:43:13.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.716+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f57cc0b43f0 con 0x7f57d00aa0c0 2024-04-25T00:43:13.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.716+0000 7f57c67fc640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f57b40678c0 0x7f57b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:13.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.716+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f57cc120010 con 0x7f57d00aa0c0 2024-04-25T00:43:13.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.716+0000 7f57d5343640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f57b40678c0 0x7f57b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:13.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.716+0000 7f57d5343640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f57b40678c0 0x7f57b4069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57d00aa9d0 tx=0x7f57c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:13.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.720+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f57cc0c4060 con 0x7f57d00aa0c0 2024-04-25T00:43:13.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.916+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57d0143950 con 0x7f57d00aa0c0 2024-04-25T00:43:13.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.916+0000 7f57c67fc640 1 -- 172.21.15.129:0/3857693556 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f57cc016110 con 0x7f57d00aa0c0 2024-04-25T00:43:13.922 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f57b40678c0 msgr2=0x7f57b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f57b40678c0 0x7f57b4069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57d00aa9d0 tx=0x7f57c0008040 comp rx=0 tx=0).stop 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 msgr2=0x7f57d0141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 0x7f57d0141a20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57cc01c350 tx=0x7f57cc01ad60 comp rx=0 tx=0).stop 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 shutdown_connections 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f57b40678c0 0x7f57b4069d80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57d00ccb50 0x7f57d0146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57d015bf50 0x7f57d01469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 --2- 172.21.15.129:0/3857693556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d00aa0c0 0x7f57d0141a20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 >> 172.21.15.129:0/3857693556 conn(0x7f57d00b14a0 msgr2=0x7f57d015a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:13.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 shutdown_connections 2024-04-25T00:43:13.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:13.920+0000 7f57d7da8640 1 -- 172.21.15.129:0/3857693556 wait complete. 2024-04-25T00:43:13.941 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:43:13.942 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T00:43:13.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:13.943 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-04-25T00:43:13.948 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:13.949 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42475sec preferred_lft 42475sec 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::ec4:7aff:fe88:7261/64 scope link 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 02:42:b7:0a:c9:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:13.950 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:43:13.951 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:13.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:13.952 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:43:14.001 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-04-25T00:43:14.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:14.002 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-04-25T00:43:14.055 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:14.056 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:43:14.056 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:14.057 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:14.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:43:14.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:43:14.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:43:14.057 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:14.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:14.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:14.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:14.179 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T00:43:14.185 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:43:14.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:14.186 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-04-25T00:43:14.231 INFO:teuthology.orchestra.run.smithi129.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T00:43:14.232 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:14.232 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:14.232 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:14.232 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T00:43:14.232 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:14.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:14.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:14.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:14.350 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T00:43:14.402 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 32:b9:ec:b6:16:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::30b9:ecff:feb6:16fb/64 scope link 2024-04-25T00:43:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:14.404 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.459 INFO:teuthology.orchestra.run.smithi129.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:43:14.460 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 32:b9:ec:b6:16:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:43:14.460 INFO:teuthology.orchestra.run.smithi129.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:43:14.460 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.460 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::30b9:ecff:feb6:16fb/64 scope link 2024-04-25T00:43:14.460 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:14.461 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T00:43:14.461 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:14.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:14.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:14.603 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:14.603 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T00:43:14.604 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.1 up 2024-04-25T00:43:14.604 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T00:43:14.604 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T00:43:14.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:14.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:14.666 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T00:43:14.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:14.666 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:14.718 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T00:43:14.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:14.718 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:14.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T00:43:14.824 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:14.872 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/run/lock 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T00:43:14.873 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/fs/nfsd 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/run/rpc_pipefs 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:14.874 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:14.875 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:14.875 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:14.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:14.875 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T00:43:14.919 INFO:teuthology.orchestra.run.smithi129.stdout:45 2024-04-25T00:43:14.919 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T00:43:14.919 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T00:43:14.964 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T00:43:14.978 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:15.027 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T00:43:15.027 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T00:43:15.027 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T00:43:15.027 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T00:43:15.027 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T00:43:15.027 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/run/lock 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/fs/nfsd 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:15.028 INFO:teuthology.orchestra.run.smithi129.stdout:/run/rpc_pipefs 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:15.029 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:15.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:15.030 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T00:43:15.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi129.stderr:2024-04-25T00:43:15.044+0000 7ff890c06580 -1 init, newargv = 0x55911f432ba0 newargc=15 2024-04-25T00:43:15.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi129.stderr:ceph-fuse[35095]: starting ceph client 2024-04-25T00:43:15.047 INFO:teuthology.orchestra.run.smithi129.stdout:45 2024-04-25T00:43:15.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi129.stderr:ceph-fuse[35095]: starting fuse 2024-04-25T00:43:16.148 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T00:43:16.164 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T00:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T00:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/run/lock 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T00:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/fs/nfsd 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/run/rpc_pipefs 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T00:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:16.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:16.216 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T00:43:16.263 INFO:teuthology.orchestra.run.smithi129.stdout:45 2024-04-25T00:43:16.263 INFO:teuthology.orchestra.run.smithi129.stdout:46 2024-04-25T00:43:16.263 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T00:43:16.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:43:16.263 DEBUG:teuthology.orchestra.run.smithi129:> import glob 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> import re 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> import os 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> import subprocess 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> def _find_admin_socket(client_name): 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> files = glob.glob(asok_path) 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> # Given a non-glob path, it better be there 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> if "*" not in asok_path: 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> assert(len(files) == 1) 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> return files[0] 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:16.264 DEBUG:teuthology.orchestra.run.smithi129:> for f in files: 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> contents = proc_f.read() 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> if mountpoint in contents: 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> return f 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> print(_find_admin_socket("client.2")) 2024-04-25T00:43:16.265 DEBUG:teuthology.orchestra.run.smithi129:> ' 2024-04-25T00:43:16.388 INFO:teuthology.orchestra.run.smithi129.stdout:/var/run/ceph/ceph-client.2.35095.asok 2024-04-25T00:43:16.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:16.400 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35095.asok 2024-04-25T00:43:16.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:16.400 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35095.asok status 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "metadata": { 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "entity_id": "2", 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": "smithi129", 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "pid": "35095", 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "root": "/" 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_count": 0, 2024-04-25T00:43:16.556 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_pinned_count": 0, 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4665, 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "inst": { 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "name": { 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "client", 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "num": 4665 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 4139126821 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T00:43:16.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 4139126821 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "inst_str": "client.4665 192.168.0.2:0/4139126821", 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "addr_str": "192.168.0.2:0/4139126821", 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "inode_count": 1, 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_epoch": 7, 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch": 20, 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklisted": false, 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs" 2024-04-25T00:43:16.558 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:43:16.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:43:16.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.688+0000 7f966d25c640 1 Processor -- start 2024-04-25T00:43:16.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.688+0000 7f966d25c640 1 -- start start 2024-04-25T00:43:16.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966d25c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 0x7f966806e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:16.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966d25c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96680b0e70 0x7f9668070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:16.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966d25c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f966806cf00 con 0x7f96680cc690 2024-04-25T00:43:16.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966d25c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f966806d1e0 con 0x7f96680b0e70 2024-04-25T00:43:16.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966d25c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f966806d4c0 con 0x7f966815e470 2024-04-25T00:43:16.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 0x7f966806e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966759e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96680b0e70 0x7f9668070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966659c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f96680cc690 0x7f96680b0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:54338/0 (socket says 172.21.15.129:54338) 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f966659c640 1 -- 172.21.15.129:0/3776374395 learned_addr learned my addr 172.21.15.129:0/3776374395 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 0x7f966806e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:45054/0 (socket says 172.21.15.129:45054) 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9665d9b640 1 -- 172.21.15.129:0/3776374395 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4150433759 0 0) 0x7f966806cf00 con 0x7f96680cc690 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9666d9d640 1 -- 172.21.15.129:0/3776374395 >> v2:172.21.15.196:3300/0 conn(0x7f96680b0e70 msgr2=0x7f9668070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9666d9d640 1 --2- 172.21.15.129:0/3776374395 >> v2:172.21.15.196:3300/0 conn(0x7f96680b0e70 0x7f9668070770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9666d9d640 1 -- 172.21.15.129:0/3776374395 >> v1:172.21.15.196:6789/0 conn(0x7f96680cc690 legacy=0x7f96680b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:16.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.692+0000 7f9666d9d640 1 -- 172.21.15.129:0/3776374395 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966806d7a0 con 0x7f966815e470 2024-04-25T00:43:16.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f9666d9d640 1 --2- 172.21.15.129:0/3776374395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 0x7f966806e280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f96580097c0 tx=0x7f965801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451ee601caa28e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f9665d9b640 1 -- 172.21.15.129:0/3776374395 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96580b0070 con 0x7f966815e470 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f9665d9b640 1 -- 172.21.15.129:0/3776374395 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96580ab600 con 0x7f966815e470 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f9665d9b640 1 -- 172.21.15.129:0/3776374395 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96580ab9d0 con 0x7f966815e470 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- 172.21.15.129:0/3776374395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 msgr2=0x7f966806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 --2- 172.21.15.129:0/3776374395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 0x7f966806e280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f96580097c0 tx=0x7f965801c410 comp rx=0 tx=0).stop 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- 172.21.15.129:0/3776374395 shutdown_connections 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 --2- 172.21.15.129:0/3776374395 >> v2:172.21.15.196:3300/0 conn(0x7f96680b0e70 0x7f9668070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 --2- 172.21.15.129:0/3776374395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815e470 0x7f966806e280 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- 172.21.15.129:0/3776374395 >> 172.21.15.129:0/3776374395 conn(0x7f96680b14a0 msgr2=0x7f96680b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:16.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- 172.21.15.129:0/3776374395 shutdown_connections 2024-04-25T00:43:16.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- 172.21.15.129:0/3776374395 wait complete. 2024-04-25T00:43:16.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 Processor -- start 2024-04-25T00:43:16.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- start start 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96680b0e70 0x7f9668152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 0x7f9668155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966815e470 0x7f96681563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9668071850 con 0x7f96680cc690 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96680716d0 con 0x7f966815e470 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966d25c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f96680719d0 con 0x7f96680b0e70 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 0x7f9668155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 0x7f9668155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55942/0 (socket says 172.21.15.129:55942) 2024-04-25T00:43:16.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 -- 172.21.15.129:0/2997425059 learned_addr learned my addr 172.21.15.129:0/2997425059 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966759e640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966815e470 0x7f96681563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f9666d9d640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96680b0e70 0x7f9668152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 -- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96680b0e70 msgr2=0x7f9668152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96680b0e70 0x7f9668152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 -- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966815e470 msgr2=0x7f96681563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966815e470 0x7f96681563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966659c640 1 -- 172.21.15.129:0/2997425059 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9668156a70 con 0x7f96680cc690 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.696+0000 7f966759e640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966815e470 0x7f96681563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f966659c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 0x7f9668155e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c450 tx=0x7f965c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c018070 con 0x7f96680cc690 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9668156d50 con 0x7f96680cc690 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965c0074e0 con 0x7f96680cc690 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0049f0 con 0x7f96680cc690 2024-04-25T00:43:16.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96681575f0 con 0x7f96680cc690 2024-04-25T00:43:16.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9668156f20 con 0x7f96680cc690 2024-04-25T00:43:16.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f965c004b90 con 0x7f96680cc690 2024-04-25T00:43:16.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f964f7fe640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96540678c0 0x7f9654069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:16.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f965c084fa0 con 0x7f96680cc690 2024-04-25T00:43:16.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f9666d9d640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96540678c0 0x7f9654069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:16.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.700+0000 7f9666d9d640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96540678c0 0x7f9654069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96580ad000 tx=0x7f96580a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:16.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.704+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f965c055af0 con 0x7f96680cc690 2024-04-25T00:43:16.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9668062e50 con 0x7f96680cc690 2024-04-25T00:43:16.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f964f7fe640 1 -- 172.21.15.129:0/2997425059 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f965c0553f0 con 0x7f96680cc690 2024-04-25T00:43:16.903 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:43:16.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96540678c0 msgr2=0x7f9654069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:16.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96540678c0 0x7f9654069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96580ad000 tx=0x7f96580a6000 comp rx=0 tx=0).stop 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 msgr2=0x7f9668155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 0x7f9668155e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c450 tx=0x7f965c00c910 comp rx=0 tx=0).stop 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 shutdown_connections 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96540678c0 0x7f9654069d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f966815e470 0x7f96681563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96680cc690 0x7f9668155e90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 --2- 172.21.15.129:0/2997425059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96680b0e70 0x7f9668152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 >> 172.21.15.129:0/2997425059 conn(0x7f96680b14a0 msgr2=0x7f96680b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 shutdown_connections 2024-04-25T00:43:16.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:16.904+0000 7f966d25c640 1 -- 172.21.15.129:0/2997425059 wait complete. 2024-04-25T00:43:16.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:43:16.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:43:16.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-25T00:43:16.922 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T00:43:16.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:16.923 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:43:16.923 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:43:16.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:43:16.923 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:43:16.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:16.923 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-25T00:43:16.930 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:16.931 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42473sec preferred_lft 42473sec 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 02:42:3b:b8:02:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:16.932 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:43:16.933 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:16.933 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr flush dev ceph-brx 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set ceph-brx up 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:43:16.933 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:17.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:17.058 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:17.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:17.065 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:43:17.075 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-25T00:43:17.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:17.076 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-25T00:43:17.142 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-25T00:43:17.142 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:43:17.142 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:17.143 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:43:17.143 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:17.144 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:17.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:43:17.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:43:17.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:43:17.144 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:17.231 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:17.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:17.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:17.263 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T00:43:17.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:17.269 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-04-25T00:43:17.318 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:17.318 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:17.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:17.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T00:43:17.318 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:17.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:17.429 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:17.436 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T00:43:17.436 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T00:43:17.437 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:17.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:17.528 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:17.535 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:17.535 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:17.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-04-25T00:43:17.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:43:17.536 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:17.615 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:17.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:17.637 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T00:43:17.637 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:17.637 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T00:43:17.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:17.683 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:17.731 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T00:43:17.788 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:17.835 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T00:43:17.835 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T00:43:17.835 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T00:43:17.835 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T00:43:17.836 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:17.837 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:17.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:17.838 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T00:43:17.886 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:43:17.886 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T00:43:17.931 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T00:43:17.945 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T00:43:17.999 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T00:43:18.000 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:43:18.001 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:43:18.001 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:43:18.001 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:43:18.001 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T00:43:18.001 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:18.001 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:18.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:18.002 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T00:43:18.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-04-25T00:43:18.008+0000 7f39ddce5580 -1 init, newargv = 0x55cbaee40930 newargc=15 2024-04-25T00:43:18.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33541]: starting ceph client 2024-04-25T00:43:18.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33541]: starting fuse 2024-04-25T00:43:19.117 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T00:43:19.132 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:19.180 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T00:43:19.180 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T00:43:19.180 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T00:43:19.180 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T00:43:19.180 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T00:43:19.181 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:19.182 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:19.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:19.183 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T00:43:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T00:43:19.227 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:19.227 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-04-25T00:43:19.228 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-04-25T00:43:19.229 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-04-25T00:43:19.229 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:43:19.229 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:19.229 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-04-25T00:43:19.229 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-04-25T00:43:19.351 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.33541.asok 2024-04-25T00:43:19.355 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:19.363 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33541.asok 2024-04-25T00:43:19.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:19.364 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33541.asok status 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "33541", 2024-04-25T00:43:19.513 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4504, 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4504 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 1966474825 2024-04-25T00:43:19.514 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 1966474825 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4504 192.168.0.1:0/1966474825", 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/1966474825", 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 7, 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 20, 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:43:19.515 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-04-25T00:43:19.516 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-04-25T00:43:19.516 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T00:43:19.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:43:19.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.655+0000 7f47f5758640 1 Processor -- start 2024-04-25T00:43:19.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.655+0000 7f47f5758640 1 -- start start 2024-04-25T00:43:19.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47f5758640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f47f015bf50 0x7f47f015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47f5758640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f47f00cd260 0x7f47f006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47f5758640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f006d1b0 con 0x7f47f00aa0c0 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47f5758640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f47f006d490 con 0x7f47f00cd260 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47f5758640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f47f006d770 con 0x7f47f015bf50 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f47f00cd260 0x7f47f006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f47f00cd260 0x7f47f006e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55952/0 (socket says 172.21.15.129:55952) 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 -- 172.21.15.129:0/1621839750 learned_addr learned my addr 172.21.15.129:0/1621839750 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47eeffd640 1 --2- 172.21.15.129:0/1621839750 >> v2:172.21.15.196:3300/0 conn(0x7f47f015bf50 0x7f47f015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 -- 172.21.15.129:0/1621839750 >> v2:172.21.15.196:3300/0 conn(0x7f47f015bf50 msgr2=0x7f47f015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 --2- 172.21.15.129:0/1621839750 >> v2:172.21.15.196:3300/0 conn(0x7f47f015bf50 0x7f47f015c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 -- 172.21.15.129:0/1621839750 >> v1:172.21.15.196:6789/0 conn(0x7f47f00aa0c0 legacy=0x7f47f00ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.659+0000 7f47ef7fe640 1 -- 172.21.15.129:0/1621839750 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f006da50 con 0x7f47f00cd260 2024-04-25T00:43:19.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ef7fe640 1 --2- 172.21.15.129:0/1621839750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f006e250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f47f0069260 tx=0x7f47e801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f7930d29d0d300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47edffb640 1 -- 172.21.15.129:0/1621839750 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80a5790 con 0x7f47f00cd260 2024-04-25T00:43:19.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47edffb640 1 -- 172.21.15.129:0/1621839750 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e80a5930 con 0x7f47f00cd260 2024-04-25T00:43:19.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47edffb640 1 -- 172.21.15.129:0/1621839750 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80a5d00 con 0x7f47f00cd260 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/1621839750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 msgr2=0x7f47f006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 --2- 172.21.15.129:0/1621839750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f006e250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f47f0069260 tx=0x7f47e801c9c0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/1621839750 shutdown_connections 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 --2- 172.21.15.129:0/1621839750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f006e250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 --2- 172.21.15.129:0/1621839750 >> v2:172.21.15.196:3300/0 conn(0x7f47f015bf50 0x7f47f015c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/1621839750 >> 172.21.15.129:0/1621839750 conn(0x7f47f00b14a0 msgr2=0x7f47f00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/1621839750 shutdown_connections 2024-04-25T00:43:19.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/1621839750 wait complete. 2024-04-25T00:43:19.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 Processor -- start 2024-04-25T00:43:19.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- start start 2024-04-25T00:43:19.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47f00aa0c0 0x7f47f0150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015bf50 0x7f47f0140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01691b0 con 0x7f47f00cd260 2024-04-25T00:43:19.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0169030 con 0x7f47f015bf50 2024-04-25T00:43:19.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f47f0169330 con 0x7f47f00aa0c0 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f014a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55958/0 (socket says 172.21.15.129:55958) 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 -- 172.21.15.129:0/3346262465 learned_addr learned my addr 172.21.15.129:0/3346262465 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47eeffd640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47f00aa0c0 0x7f47f0150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ef7fe640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015bf50 0x7f47f0140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 -- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47f00aa0c0 msgr2=0x7f47f0150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47f00aa0c0 0x7f47f0150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 -- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015bf50 msgr2=0x7f47f0140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015bf50 0x7f47f0140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 -- 172.21.15.129:0/3346262465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0154750 con 0x7f47f00cd260 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47ee7fc640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f014a8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c500 tx=0x7f47e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4016020 con 0x7f47f00cd260 2024-04-25T00:43:19.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e4005220 con 0x7f47f00cd260 2024-04-25T00:43:19.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f0154a30 con 0x7f47f00cd260 2024-04-25T00:43:19.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4007030 con 0x7f47f00cd260 2024-04-25T00:43:19.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.663+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f0154ff0 con 0x7f47f00cd260 2024-04-25T00:43:19.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.667+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0062e50 con 0x7f47f00cd260 2024-04-25T00:43:19.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.667+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47e40071f0 con 0x7f47f00cd260 2024-04-25T00:43:19.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.667+0000 7f47d77fe640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f47dc0678c0 0x7f47dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.667+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47e40856b0 con 0x7f47f00cd260 2024-04-25T00:43:19.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.667+0000 7f47eeffd640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f47dc0678c0 0x7f47dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.667+0000 7f47eeffd640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f47dc0678c0 0x7f47dc069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47e00096f0 tx=0x7f47e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.671+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47e4020020 con 0x7f47f00cd260 2024-04-25T00:43:19.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.863+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f47f0075320 con 0x7f47f00cd260 2024-04-25T00:43:19.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47d77fe640 1 -- 172.21.15.129:0/3346262465 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f47e4056200 con 0x7f47f00cd260 2024-04-25T00:43:19.867 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:43:19.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f47dc0678c0 msgr2=0x7f47dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f47dc0678c0 0x7f47dc069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47e00096f0 tx=0x7f47e0008040 comp rx=0 tx=0).stop 2024-04-25T00:43:19.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 msgr2=0x7f47f014a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f014a8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c500 tx=0x7f47e400c9c0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 shutdown_connections 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f47dc0678c0 0x7f47dc069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015bf50 0x7f47f0140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47f00cd260 0x7f47f014a8c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 --2- 172.21.15.129:0/3346262465 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47f00aa0c0 0x7f47f0150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 >> 172.21.15.129:0/3346262465 conn(0x7f47f00b14a0 msgr2=0x7f47f00a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 shutdown_connections 2024-04-25T00:43:19.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:43:19.867+0000 7f47f5758640 1 -- 172.21.15.129:0/3346262465 wait complete. 2024-04-25T00:43:19.885 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:43:19.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:43:19.886 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T00:43:19.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:19.886 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-25T00:43:19.893 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:19.893 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:19.894 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42470sec preferred_lft 42470sec 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 02:42:3b:b8:02:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:43:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.896 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:43:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:43:19.896 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:19.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:19.896 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:43:19.949 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-25T00:43:19.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:19.951 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:20.003 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:43:20.004 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:43:20.004 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:20.004 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:20.004 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:43:20.004 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:43:20.004 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:43:20.004 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:20.092 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:20.119 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:20.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:20.128 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T00:43:20.133 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:43:20.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:20.134 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-04-25T00:43:20.183 INFO:teuthology.orchestra.run.smithi196.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T00:43:20.183 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:20.183 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:20.183 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:20.183 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T00:43:20.183 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:20.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:20.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:20.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:20.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 6a:dc:6a:1d:0a:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:43:20.314 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:43:20.315 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.315 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::68dc:6aff:fe1d:adb/64 scope link 2024-04-25T00:43:20.315 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:20.316 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 6a:dc:6a:1d:0a:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T00:43:20.368 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T00:43:20.369 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.369 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::68dc:6aff:fe1d:adb/64 scope link 2024-04-25T00:43:20.369 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:43:20.370 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:20.455 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:20.507 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:20.515 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:43:20.515 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T00:43:20.515 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.1 up 2024-04-25T00:43:20.515 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T00:43:20.515 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T00:43:20.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:43:20.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:20.622 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T00:43:20.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:20.622 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:20.671 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T00:43:20.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:43:20.672 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:20.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T00:43:20.776 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T00:43:20.827 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T00:43:20.828 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:20.829 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:20.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:20.830 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T00:43:20.879 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T00:43:20.879 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T00:43:20.879 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T00:43:20.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T00:43:20.936 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:20.983 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T00:43:20.983 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T00:43:20.983 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T00:43:20.983 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:20.984 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T00:43:20.985 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:20.986 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:20.987 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:20.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:20.987 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T00:43:21.006 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:2024-04-25T00:43:21.000+0000 7f06a2408580 -1 init, newargv = 0x562744d78ae0 newargc=15 2024-04-25T00:43:21.006 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33660]: starting ceph client 2024-04-25T00:43:21.010 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T00:43:21.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33660]: starting fuse 2024-04-25T00:43:22.112 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T00:43:22.128 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:43:22.176 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T00:43:22.176 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T00:43:22.176 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:43:22.177 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T00:43:22.178 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:22.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.179 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T00:43:22.223 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T00:43:22.223 INFO:teuthology.orchestra.run.smithi196.stdout:46 2024-04-25T00:43:22.223 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T00:43:22.223 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:43:22.223 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:43:22.224 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.3")) 2024-04-25T00:43:22.225 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-04-25T00:43:22.347 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.3.33660.asok 2024-04-25T00:43:22.351 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T00:43:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:43:22.359 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33660.asok 2024-04-25T00:43:22.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.359 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33660.asok status 2024-04-25T00:43:22.511 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "3", 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T00:43:22.512 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "33660", 2024-04-25T00:43:22.513 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-04-25T00:43:22.513 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:22.513 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-04-25T00:43:22.513 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-04-25T00:43:22.513 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4516, 2024-04-25T00:43:22.513 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4516 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:43:22.514 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 645919048 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.2:0", 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 645919048 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4516 192.168.0.2:0/645919048", 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.2:0/645919048", 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-04-25T00:43:22.515 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 7, 2024-04-25T00:43:22.516 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 20, 2024-04-25T00:43:22.516 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:43:22.516 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-04-25T00:43:22.516 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-04-25T00:43:22.516 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T00:43:22.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.526 DEBUG:teuthology.orchestra.run.smithi129:> stat --file-system '--printf=%T 2024-04-25T00:43:22.526 DEBUG:teuthology.orchestra.run.smithi129:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:22.534 INFO:teuthology.orchestra.run.smithi129.stdout:fuseblk 2024-04-25T00:43:22.534 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:22.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.534 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:22.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.591 DEBUG:teuthology.orchestra.run.smithi129:> stat --file-system '--printf=%T 2024-04-25T00:43:22.591 DEBUG:teuthology.orchestra.run.smithi129:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:22.645 INFO:teuthology.orchestra.run.smithi129.stdout:fuseblk 2024-04-25T00:43:22.645 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:22.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T00:43:22.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.706 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-04-25T00:43:22.706 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:22.713 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-04-25T00:43:22.713 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:22.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.713 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T00:43:22.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.767 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-04-25T00:43:22.767 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:22.816 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-04-25T00:43:22.816 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:22.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:43:22.816 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T00:43:22.871 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T00:43:22.883 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:43:22.884 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T00:43:22.884 DEBUG:teuthology.orchestra.run.smithi129:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:22.892 INFO:teuthology.orchestra.run.smithi129.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:22.892 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T00:43:22.892 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 2dh/45d Inode: 1 Links: 2 2024-04-25T00:43:22.892 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:43:22.892 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T00:43:22.892 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:42:58.651149876 +0000 2024-04-25T00:43:22.893 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:43:22.588196269 +0000 2024-04-25T00:43:22.893 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2024-04-25T00:43:22.893 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T00:43:22.893 DEBUG:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T00:43:22.958 INFO:tasks.workunit:timeout=3h 2024-04-25T00:43:22.958 INFO:tasks.workunit:cleanup=True 2024-04-25T00:43:22.959 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:43:23.009 INFO:tasks.workunit.client.0.smithi129.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T00:43:27.471 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:43:27.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:43:57.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:43:57.580 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:44:27.623 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:44:27.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.140 INFO:tasks.workunit.client.0.smithi129.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: git switch -c 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr:Or undo this operation with: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: git switch - 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T00:44:31.141 INFO:tasks.workunit.client.0.smithi129.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T00:44:31.144 DEBUG:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T00:44:31.151 INFO:tasks.workunit.client.0.smithi129.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T00:44:31.155 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:44:31.155 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T00:44:31.974 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T00:44:32.056 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T00:44:32.114 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:44:32.115 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:44:32.116 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T00:44:32.168 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:44:32.172 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:44:32.172 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T00:44:32.220 INFO:tasks.workunit:Running workunits matching fs/maxentries on client.0... 2024-04-25T00:44:32.222 INFO:tasks.workunit:Running workunit fs/maxentries/maxentries.sh... 2024-04-25T00:44:32.222 DEBUG:teuthology.orchestra.run.smithi129:workunit test fs/maxentries/maxentries.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/maxentries/maxentries.sh 2024-04-25T00:44:32.286 INFO:tasks.workunit.client.0.smithi129.stderr:+ test_dir=max_entries 2024-04-25T00:44:32.286 INFO:tasks.workunit.client.0.smithi129.stderr:+ mkdir -p max_entries 2024-04-25T00:44:32.292 INFO:tasks.workunit.client.0.smithi129.stderr:+ pushd max_entries 2024-04-25T00:44:32.293 INFO:tasks.workunit.client.0.smithi129.stdout:~/cephtest/mnt.0/client.0/tmp/max_entries ~/cephtest/mnt.0/client.0/tmp 2024-04-25T00:44:32.294 INFO:tasks.workunit.client.0.smithi129.stderr:+ dir_max_entries=100 2024-04-25T00:44:32.294 INFO:tasks.workunit.client.0.smithi129.stderr:+ ceph config set mds mds_dir_max_entries 100 2024-04-25T00:44:32.412 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0c5bf640 1 Processor -- start 2024-04-25T00:44:32.413 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0c5bf640 1 -- start start 2024-04-25T00:44:32.416 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0c5bf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 0x7fba0406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0c5bf640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba040ce120 con 0x7fba0406f540 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0c5bf640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba040ce400 con 0x7fba040cad60 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0c5bf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba040ce6e0 con 0x7fba040cb140 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09b5a640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fba0406f540 0x7fba0406c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:43604/0 (socket says 172.21.15.129:43604) 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09b5a640 1 -- 172.21.15.129:0/3702943090 learned_addr learned my addr 172.21.15.129:0/3702943090 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 0x7fba0406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2228323770 0 0) 0x7fba040ce120 con 0x7fba0406f540 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9f0003530 con 0x7fba0406f540 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2176295565 0 0) 0x7fb9f0003530 con 0x7fba0406f540 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 msgr2=0x7fba0406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:32.417 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 0x7fba0406d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 >> v1:172.21.15.196:6789/0 conn(0x7fba040cad60 legacy=0x7fba0406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba040ce9c0 con 0x7fba0406f540 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fb9f4001680 con 0x7fba0406f540 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 >> v1:172.21.15.129:6789/0 conn(0x7fba0406f540 legacy=0x7fba0406c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 0x7fb9f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f0004d50 0x7fb9f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba040ce6e0 con 0x7fb9f0004d50 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f000bab0 con 0x7fb9f00039f0 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f000bcb0 con 0x7fba040cb140 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f0004d50 0x7fb9f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.418 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0a35b640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba09b5a640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 0x7fb9f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 -- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 msgr2=0x7fb9f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040cb140 0x7fb9f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 -- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f00039f0 msgr2=0x7fb9f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0ab5c640 1 -- 172.21.15.129:0/3702943090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f0003530 con 0x7fb9f0004d50 2024-04-25T00:44:32.419 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.409+0000 7fba0a35b640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f0004d50 0x7fb9f00051d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f90 tx=0x7fb9fc01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b1161bed83c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0b2070 con 0x7fb9f0004d50 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc0a5c50 con 0x7fb9f0004d50 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba09359640 1 -- 172.21.15.129:0/3702943090 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc009c10 con 0x7fb9f0004d50 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 -- 172.21.15.129:0/3702943090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f0004d50 msgr2=0x7fb9f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/3702943090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f0004d50 0x7fb9f00051d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f90 tx=0x7fb9fc01cc40 comp rx=0 tx=0).stop 2024-04-25T00:44:32.423 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0ab5c640 1 -- 172.21.15.129:0/3702943090 reap_dead start 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 -- 172.21.15.129:0/3702943090 shutdown_connections 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 -- 172.21.15.129:0/3702943090 >> 172.21.15.129:0/3702943090 conn(0x7fba040ccf30 msgr2=0x7fba040cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 -- 172.21.15.129:0/3702943090 shutdown_connections 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 -- 172.21.15.129:0/3702943090 wait complete. 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 Processor -- start 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.413+0000 7fba0c5bf640 1 -- start start 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0414b7c0 0x7fba040ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 0x7fba04159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040c8d10 0x7fba0415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba0406c7c0 con 0x7fba0414bba0 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba040ceca0 con 0x7fba0414b7c0 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fba0406c940 con 0x7fba040c8d10 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 0x7fba04159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 0x7fba04159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51506/0 (socket says 172.21.15.129:51506) 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 -- 172.21.15.129:0/1622380491 learned_addr learned my addr 172.21.15.129:0/1622380491 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:44:32.424 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040c8d10 0x7fba0415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0a35b640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0414b7c0 0x7fba040ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 -- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040c8d10 msgr2=0x7fba0415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040c8d10 0x7fba0415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 -- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0414b7c0 msgr2=0x7fba040ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0414b7c0 0x7fba040ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 -- 172.21.15.129:0/1622380491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba040c9280 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0ab5c640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040c8d10 0x7fba0415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba09b5a640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 0x7fba04159660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400eb20 tx=0x7fb9f40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400c850 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f4018070 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba040c9560 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400cc30 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba040c9b20 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba04068e70 con 0x7fba0414bba0 2024-04-25T00:44:32.425 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb9f4013420 con 0x7fba0414bba0 2024-04-25T00:44:32.426 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fb9faffd640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9e80678f0 0x7fb9e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:32.426 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb9f4085a00 con 0x7fba0414bba0 2024-04-25T00:44:32.426 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0a35b640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9e80678f0 0x7fb9e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:32.426 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.417+0000 7fba0a35b640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9e80678f0 0x7fb9e8069db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba00000c10 tx=0x7fba00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:32.427 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.421+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb9f40563f0 con 0x7fba0414bba0 2024-04-25T00:44:32.588 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.581+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mds_dir_max_entries}] v 0) -- 0x7fba0406b540 con 0x7fba0414bba0 2024-04-25T00:44:32.592 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.589+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_dir_max_entries}]=0 v10)=0 v10) ==== 119+0+0 (secure 0 0 0) 0x7fb9f4055cf0 con 0x7fba0414bba0 2024-04-25T00:44:32.691 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.685+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mds_dir_max_entries}] v 0) -- 0x7fba04159ba0 con 0x7fba0414bba0 2024-04-25T00:44:32.691 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.685+0000 7fb9faffd640 1 -- 172.21.15.129:0/1622380491 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_dir_max_entries}]=0 v10)=0 v10) ==== 119+0+0 (secure 0 0 0) 0x7fb9f405a0a0 con 0x7fba0414bba0 2024-04-25T00:44:32.695 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9e80678f0 msgr2=0x7fb9e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.695 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9e80678f0 0x7fb9e8069db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba00000c10 tx=0x7fba00009040 comp rx=0 tx=0).stop 2024-04-25T00:44:32.695 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 msgr2=0x7fba04159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:32.695 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 0x7fba04159660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400eb20 tx=0x7fb9f40016f0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 shutdown_connections 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb9e80678f0 0x7fb9e8069db0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba040c8d10 0x7fba0415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba0414bba0 0x7fba04159660 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 --2- 172.21.15.129:0/1622380491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba0414b7c0 0x7fba040ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 >> 172.21.15.129:0/1622380491 conn(0x7fba040ccf30 msgr2=0x7fba0406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 shutdown_connections 2024-04-25T00:44:32.696 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:32.689+0000 7fba0c5bf640 1 -- 172.21.15.129:0/1622380491 wait complete. 2024-04-25T00:44:32.710 INFO:tasks.workunit.client.0.smithi129.stderr:+ ok_dir_max_entries=100 2024-04-25T00:44:32.711 INFO:tasks.workunit.client.0.smithi129.stderr:+ fail_dir_max_entries=101 2024-04-25T00:44:32.711 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_files 100 2024-04-25T00:44:32.711 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:32.729 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.729 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.729 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00882927 s, 0.0 kB/s 2024-04-25T00:44:32.755 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.755 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.755 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00591044 s, 0.0 kB/s 2024-04-25T00:44:32.763 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.763 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.763 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00362138 s, 0.0 kB/s 2024-04-25T00:44:32.774 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.774 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.774 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00599127 s, 0.0 kB/s 2024-04-25T00:44:32.783 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.784 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.784 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0043607 s, 0.0 kB/s 2024-04-25T00:44:32.795 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.795 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.795 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00646843 s, 0.0 kB/s 2024-04-25T00:44:32.804 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.804 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.804 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00429829 s, 0.0 kB/s 2024-04-25T00:44:32.816 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.816 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.816 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00664118 s, 0.0 kB/s 2024-04-25T00:44:32.825 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.825 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.825 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00386512 s, 0.0 kB/s 2024-04-25T00:44:32.833 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.833 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.833 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00346883 s, 0.0 kB/s 2024-04-25T00:44:32.842 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.842 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.842 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00392528 s, 0.0 kB/s 2024-04-25T00:44:32.851 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.851 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.851 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379741 s, 0.0 kB/s 2024-04-25T00:44:32.860 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.860 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.860 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00345034 s, 0.0 kB/s 2024-04-25T00:44:32.869 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.869 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.869 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376164 s, 0.0 kB/s 2024-04-25T00:44:32.877 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.877 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.877 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00342989 s, 0.0 kB/s 2024-04-25T00:44:32.887 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.887 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.887 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00447145 s, 0.0 kB/s 2024-04-25T00:44:32.895 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.895 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.896 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00344834 s, 0.0 kB/s 2024-04-25T00:44:32.907 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.907 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.907 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00610767 s, 0.0 kB/s 2024-04-25T00:44:32.916 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.916 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.916 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00336133 s, 0.0 kB/s 2024-04-25T00:44:32.927 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.928 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.928 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00605621 s, 0.0 kB/s 2024-04-25T00:44:32.936 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.936 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.936 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00349046 s, 0.0 kB/s 2024-04-25T00:44:32.948 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.948 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.948 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0062724 s, 0.0 kB/s 2024-04-25T00:44:32.956 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.956 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.956 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00362332 s, 0.0 kB/s 2024-04-25T00:44:32.968 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.968 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.968 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0062705 s, 0.0 kB/s 2024-04-25T00:44:32.977 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.977 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.977 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377836 s, 0.0 kB/s 2024-04-25T00:44:32.988 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.988 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.988 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00605695 s, 0.0 kB/s 2024-04-25T00:44:32.997 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:32.997 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:32.997 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00375091 s, 0.0 kB/s 2024-04-25T00:44:33.010 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.010 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.010 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00642125 s, 0.0 kB/s 2024-04-25T00:44:33.020 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.020 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.020 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00352102 s, 0.0 kB/s 2024-04-25T00:44:33.032 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.032 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.033 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00594764 s, 0.0 kB/s 2024-04-25T00:44:33.042 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.043 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.043 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388819 s, 0.0 kB/s 2024-04-25T00:44:33.056 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.056 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.056 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00630024 s, 0.0 kB/s 2024-04-25T00:44:33.066 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.066 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.067 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00367079 s, 0.0 kB/s 2024-04-25T00:44:33.079 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.079 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.080 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00640605 s, 0.0 kB/s 2024-04-25T00:44:33.090 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.090 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.090 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0036469 s, 0.0 kB/s 2024-04-25T00:44:33.100 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.100 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.100 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376545 s, 0.0 kB/s 2024-04-25T00:44:33.110 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.110 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.110 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0034284 s, 0.0 kB/s 2024-04-25T00:44:33.122 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.123 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.123 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00592786 s, 0.0 kB/s 2024-04-25T00:44:33.133 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.133 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.133 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00357777 s, 0.0 kB/s 2024-04-25T00:44:33.145 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.146 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.146 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00605181 s, 0.0 kB/s 2024-04-25T00:44:33.157 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.157 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.157 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00427326 s, 0.0 kB/s 2024-04-25T00:44:33.168 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.168 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.168 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00584193 s, 0.0 kB/s 2024-04-25T00:44:33.177 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.177 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.177 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00385408 s, 0.0 kB/s 2024-04-25T00:44:33.188 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.188 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.188 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00575797 s, 0.0 kB/s 2024-04-25T00:44:33.198 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.198 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.198 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00438606 s, 0.0 kB/s 2024-04-25T00:44:33.207 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.207 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.207 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00442243 s, 0.0 kB/s 2024-04-25T00:44:33.217 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.217 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.217 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00440066 s, 0.0 kB/s 2024-04-25T00:44:33.226 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.227 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00441391 s, 0.0 kB/s 2024-04-25T00:44:33.236 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.236 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.236 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381738 s, 0.0 kB/s 2024-04-25T00:44:33.246 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.246 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.246 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00437392 s, 0.0 kB/s 2024-04-25T00:44:33.255 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.255 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.255 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00432559 s, 0.0 kB/s 2024-04-25T00:44:33.265 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.265 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.265 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00439971 s, 0.0 kB/s 2024-04-25T00:44:33.274 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.274 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.274 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381431 s, 0.0 kB/s 2024-04-25T00:44:33.283 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.283 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.283 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389266 s, 0.0 kB/s 2024-04-25T00:44:33.292 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.292 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.292 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00380027 s, 0.0 kB/s 2024-04-25T00:44:33.301 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.301 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.302 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00415207 s, 0.0 kB/s 2024-04-25T00:44:33.311 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.311 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.311 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389143 s, 0.0 kB/s 2024-04-25T00:44:33.321 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.321 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.321 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0045584 s, 0.0 kB/s 2024-04-25T00:44:33.330 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.330 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.331 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00437966 s, 0.0 kB/s 2024-04-25T00:44:33.341 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.342 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.342 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0058154 s, 0.0 kB/s 2024-04-25T00:44:33.350 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.351 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.351 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00378296 s, 0.0 kB/s 2024-04-25T00:44:33.362 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.362 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.362 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00579562 s, 0.0 kB/s 2024-04-25T00:44:33.371 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.371 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.371 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381029 s, 0.0 kB/s 2024-04-25T00:44:33.383 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.383 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.383 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0060546 s, 0.0 kB/s 2024-04-25T00:44:33.393 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.393 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.393 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00430667 s, 0.0 kB/s 2024-04-25T00:44:33.405 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.405 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.405 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00594781 s, 0.0 kB/s 2024-04-25T00:44:33.414 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.414 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.414 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00375032 s, 0.0 kB/s 2024-04-25T00:44:33.425 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.425 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.425 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00611177 s, 0.0 kB/s 2024-04-25T00:44:33.434 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.434 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.434 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00368171 s, 0.0 kB/s 2024-04-25T00:44:33.446 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.446 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.446 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00617479 s, 0.0 kB/s 2024-04-25T00:44:33.454 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.454 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.454 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00370922 s, 0.0 kB/s 2024-04-25T00:44:33.466 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.466 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.466 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00602624 s, 0.0 kB/s 2024-04-25T00:44:33.475 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.475 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.475 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399668 s, 0.0 kB/s 2024-04-25T00:44:33.487 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.487 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.487 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00625261 s, 0.0 kB/s 2024-04-25T00:44:33.498 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.498 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.498 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00430034 s, 0.0 kB/s 2024-04-25T00:44:33.507 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.507 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.507 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00429574 s, 0.0 kB/s 2024-04-25T00:44:33.516 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.516 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.516 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00369728 s, 0.0 kB/s 2024-04-25T00:44:33.526 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.526 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.526 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00428877 s, 0.0 kB/s 2024-04-25T00:44:33.535 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.535 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.535 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00375814 s, 0.0 kB/s 2024-04-25T00:44:33.544 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.544 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.545 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404618 s, 0.0 kB/s 2024-04-25T00:44:33.554 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.554 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.554 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394714 s, 0.0 kB/s 2024-04-25T00:44:33.563 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.563 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.563 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400605 s, 0.0 kB/s 2024-04-25T00:44:33.572 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.572 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.572 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00373819 s, 0.0 kB/s 2024-04-25T00:44:33.582 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.582 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.582 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00435748 s, 0.0 kB/s 2024-04-25T00:44:33.591 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.591 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.591 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400987 s, 0.0 kB/s 2024-04-25T00:44:33.602 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.602 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.602 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426009 s, 0.0 kB/s 2024-04-25T00:44:33.611 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.611 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.611 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00357102 s, 0.0 kB/s 2024-04-25T00:44:33.620 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.620 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.621 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00463528 s, 0.0 kB/s 2024-04-25T00:44:33.630 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.630 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.630 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00390318 s, 0.0 kB/s 2024-04-25T00:44:33.639 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.640 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.640 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00455976 s, 0.0 kB/s 2024-04-25T00:44:33.649 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.649 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.649 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394402 s, 0.0 kB/s 2024-04-25T00:44:33.659 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.659 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.659 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0045704 s, 0.0 kB/s 2024-04-25T00:44:33.669 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.669 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.669 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410846 s, 0.0 kB/s 2024-04-25T00:44:33.678 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.678 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.678 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379502 s, 0.0 kB/s 2024-04-25T00:44:33.689 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.689 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.689 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418481 s, 0.0 kB/s 2024-04-25T00:44:33.697 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.697 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.697 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00343632 s, 0.0 kB/s 2024-04-25T00:44:33.707 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.707 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.707 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00433949 s, 0.0 kB/s 2024-04-25T00:44:33.715 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.715 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.715 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00360683 s, 0.0 kB/s 2024-04-25T00:44:33.728 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.728 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.728 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00657721 s, 0.0 kB/s 2024-04-25T00:44:33.737 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.737 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.737 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00447471 s, 0.0 kB/s 2024-04-25T00:44:33.739 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:33.739 INFO:tasks.workunit.client.0.smithi129.stderr:+ expect_false make_files 101 2024-04-25T00:44:33.739 INFO:tasks.workunit.client.0.smithi129.stderr:+ set -x 2024-04-25T00:44:33.739 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_files 101 2024-04-25T00:44:33.739 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:33.752 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.752 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.752 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00464246 s, 0.0 kB/s 2024-04-25T00:44:33.761 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.761 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.761 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00371573 s, 0.0 kB/s 2024-04-25T00:44:33.770 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.770 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.770 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00372947 s, 0.0 kB/s 2024-04-25T00:44:33.779 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.779 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.779 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00390865 s, 0.0 kB/s 2024-04-25T00:44:33.788 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.788 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.788 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00356454 s, 0.0 kB/s 2024-04-25T00:44:33.796 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.797 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.797 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00384086 s, 0.0 kB/s 2024-04-25T00:44:33.805 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.805 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.806 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387666 s, 0.0 kB/s 2024-04-25T00:44:33.814 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.814 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.814 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00363101 s, 0.0 kB/s 2024-04-25T00:44:33.823 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.823 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.823 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377769 s, 0.0 kB/s 2024-04-25T00:44:33.831 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.832 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.832 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00344305 s, 0.0 kB/s 2024-04-25T00:44:33.841 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.841 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.841 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0043139 s, 0.0 kB/s 2024-04-25T00:44:33.850 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.850 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.850 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00372142 s, 0.0 kB/s 2024-04-25T00:44:33.859 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.859 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.859 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00431719 s, 0.0 kB/s 2024-04-25T00:44:33.869 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.869 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.869 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00520929 s, 0.0 kB/s 2024-04-25T00:44:33.878 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.878 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.878 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426901 s, 0.0 kB/s 2024-04-25T00:44:33.887 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.887 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.887 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00384703 s, 0.0 kB/s 2024-04-25T00:44:33.897 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.897 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.897 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0043268 s, 0.0 kB/s 2024-04-25T00:44:33.906 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.906 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.906 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383656 s, 0.0 kB/s 2024-04-25T00:44:33.915 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.915 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.915 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00435701 s, 0.0 kB/s 2024-04-25T00:44:33.924 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.924 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.924 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0035075 s, 0.0 kB/s 2024-04-25T00:44:33.933 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.933 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.933 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00432693 s, 0.0 kB/s 2024-04-25T00:44:33.942 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.942 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.942 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00372032 s, 0.0 kB/s 2024-04-25T00:44:33.951 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.951 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.951 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379922 s, 0.0 kB/s 2024-04-25T00:44:33.959 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.959 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.959 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00348459 s, 0.0 kB/s 2024-04-25T00:44:33.968 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.968 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.968 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00374023 s, 0.0 kB/s 2024-04-25T00:44:33.977 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.977 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.977 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00378381 s, 0.0 kB/s 2024-04-25T00:44:33.986 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.986 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.986 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387879 s, 0.0 kB/s 2024-04-25T00:44:33.994 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:33.994 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:33.995 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00369437 s, 0.0 kB/s 2024-04-25T00:44:34.003 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.004 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.004 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410944 s, 0.0 kB/s 2024-04-25T00:44:34.012 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.012 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.012 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00361341 s, 0.0 kB/s 2024-04-25T00:44:34.024 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.024 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.024 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00633204 s, 0.0 kB/s 2024-04-25T00:44:34.033 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.033 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.033 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376493 s, 0.0 kB/s 2024-04-25T00:44:34.044 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.044 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.044 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00642797 s, 0.0 kB/s 2024-04-25T00:44:34.053 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.053 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.053 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00368286 s, 0.0 kB/s 2024-04-25T00:44:34.065 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.065 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.065 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00660321 s, 0.0 kB/s 2024-04-25T00:44:34.076 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.076 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.076 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00616828 s, 0.0 kB/s 2024-04-25T00:44:34.089 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.089 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.089 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00662287 s, 0.0 kB/s 2024-04-25T00:44:34.097 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.097 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.098 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00359294 s, 0.0 kB/s 2024-04-25T00:44:34.110 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.110 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.110 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00621537 s, 0.0 kB/s 2024-04-25T00:44:34.118 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.118 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.118 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00385014 s, 0.0 kB/s 2024-04-25T00:44:34.128 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.129 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.129 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00576573 s, 0.0 kB/s 2024-04-25T00:44:34.137 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.137 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.137 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00374973 s, 0.0 kB/s 2024-04-25T00:44:34.146 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.146 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.146 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00351374 s, 0.0 kB/s 2024-04-25T00:44:34.155 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.155 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.155 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406298 s, 0.0 kB/s 2024-04-25T00:44:34.164 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.164 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.164 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377992 s, 0.0 kB/s 2024-04-25T00:44:34.173 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.173 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.173 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407584 s, 0.0 kB/s 2024-04-25T00:44:34.182 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.182 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.182 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0037434 s, 0.0 kB/s 2024-04-25T00:44:34.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.191 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038702 s, 0.0 kB/s 2024-04-25T00:44:34.200 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.200 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.200 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00391441 s, 0.0 kB/s 2024-04-25T00:44:34.209 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.209 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.209 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404031 s, 0.0 kB/s 2024-04-25T00:44:34.218 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.218 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.218 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383418 s, 0.0 kB/s 2024-04-25T00:44:34.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.227 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387423 s, 0.0 kB/s 2024-04-25T00:44:34.238 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.238 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.238 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00623586 s, 0.0 kB/s 2024-04-25T00:44:34.247 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.247 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.247 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388363 s, 0.0 kB/s 2024-04-25T00:44:34.259 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.259 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.259 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00610804 s, 0.0 kB/s 2024-04-25T00:44:34.268 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.268 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.268 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397962 s, 0.0 kB/s 2024-04-25T00:44:34.279 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.279 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.279 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00602391 s, 0.0 kB/s 2024-04-25T00:44:34.288 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.288 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.288 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379128 s, 0.0 kB/s 2024-04-25T00:44:34.299 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.299 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.299 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00610389 s, 0.0 kB/s 2024-04-25T00:44:34.309 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.309 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.309 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00436917 s, 0.0 kB/s 2024-04-25T00:44:34.320 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.320 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.320 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00620362 s, 0.0 kB/s 2024-04-25T00:44:34.329 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.329 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.330 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406713 s, 0.0 kB/s 2024-04-25T00:44:34.340 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.340 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.341 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00581326 s, 0.0 kB/s 2024-04-25T00:44:34.349 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.349 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.349 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00375866 s, 0.0 kB/s 2024-04-25T00:44:34.357 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.358 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.358 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00365283 s, 0.0 kB/s 2024-04-25T00:44:34.366 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.367 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.367 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381988 s, 0.0 kB/s 2024-04-25T00:44:34.376 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.376 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.376 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399351 s, 0.0 kB/s 2024-04-25T00:44:34.385 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.385 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.385 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377858 s, 0.0 kB/s 2024-04-25T00:44:34.394 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.394 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.394 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0041186 s, 0.0 kB/s 2024-04-25T00:44:34.403 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.404 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.404 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00433517 s, 0.0 kB/s 2024-04-25T00:44:34.413 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.413 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.413 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00402966 s, 0.0 kB/s 2024-04-25T00:44:34.421 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.422 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.422 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00365024 s, 0.0 kB/s 2024-04-25T00:44:34.430 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.431 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.431 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00390738 s, 0.0 kB/s 2024-04-25T00:44:34.439 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.439 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.440 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00373025 s, 0.0 kB/s 2024-04-25T00:44:34.451 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.451 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.451 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00617473 s, 0.0 kB/s 2024-04-25T00:44:34.460 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.460 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.460 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00361397 s, 0.0 kB/s 2024-04-25T00:44:34.469 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.469 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.469 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00441238 s, 0.0 kB/s 2024-04-25T00:44:34.478 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.536 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.536 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379017 s, 0.0 kB/s 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426004 s, 0.0 kB/s 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00360935 s, 0.0 kB/s 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00432694 s, 0.0 kB/s 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.538 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00357125 s, 0.0 kB/s 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00447333 s, 0.0 kB/s 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.539 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389 s, 0.0 kB/s 2024-04-25T00:44:34.542 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.542 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.542 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00429365 s, 0.0 kB/s 2024-04-25T00:44:34.551 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.551 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.551 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00366845 s, 0.0 kB/s 2024-04-25T00:44:34.560 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.560 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.560 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00391767 s, 0.0 kB/s 2024-04-25T00:44:34.569 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.569 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.569 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00370645 s, 0.0 kB/s 2024-04-25T00:44:34.578 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.579 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.579 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383008 s, 0.0 kB/s 2024-04-25T00:44:34.587 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.587 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.587 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0036051 s, 0.0 kB/s 2024-04-25T00:44:34.597 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.598 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.598 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0055754 s, 0.0 kB/s 2024-04-25T00:44:34.606 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.607 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.607 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00373952 s, 0.0 kB/s 2024-04-25T00:44:34.615 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.615 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.615 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00361054 s, 0.0 kB/s 2024-04-25T00:44:34.624 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.624 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.624 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038801 s, 0.0 kB/s 2024-04-25T00:44:34.634 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.634 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.634 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388094 s, 0.0 kB/s 2024-04-25T00:44:34.643 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.643 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.643 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401567 s, 0.0 kB/s 2024-04-25T00:44:34.654 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.654 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.654 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0059744 s, 0.0 kB/s 2024-04-25T00:44:34.663 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.663 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.663 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397209 s, 0.0 kB/s 2024-04-25T00:44:34.675 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.675 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.675 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0062743 s, 0.0 kB/s 2024-04-25T00:44:34.686 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:34.686 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:34.687 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00609528 s, 0.0 kB/s 2024-04-25T00:44:34.691 INFO:tasks.workunit.client.0.smithi129.stderr:dd: failed to open 'mkfile_test_8q3TM3/file_101': No space left on device 2024-04-25T00:44:35.594 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:35.594 INFO:tasks.workunit.client.0.smithi129.stderr:+ expect_false make_dirs 101 2024-04-25T00:44:35.594 INFO:tasks.workunit.client.0.smithi129.stderr:+ set -x 2024-04-25T00:44:35.594 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_dirs 101 2024-04-25T00:44:35.594 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:36.495 INFO:tasks.workunit.client.0.smithi129.stderr:mkdir: cannot create directory 'mkdir_test_nYz8lE/dir_101': No space left on device 2024-04-25T00:44:37.373 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:37.373 INFO:tasks.workunit.client.0.smithi129.stderr:+ expect_false make_nodes 101 2024-04-25T00:44:37.373 INFO:tasks.workunit.client.0.smithi129.stderr:+ set -x 2024-04-25T00:44:37.374 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_nodes 101 2024-04-25T00:44:37.374 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:38.351 INFO:tasks.workunit.client.0.smithi129.stderr:mknod: mknod_test_SSBadR/fifo_101: No space left on device 2024-04-25T00:44:40.268 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:40.268 INFO:tasks.workunit.client.0.smithi129.stderr:+ expect_false rename_files 101 2024-04-25T00:44:40.269 INFO:tasks.workunit.client.0.smithi129.stderr:+ set -x 2024-04-25T00:44:40.269 INFO:tasks.workunit.client.0.smithi129.stderr:+ rename_files 101 2024-04-25T00:44:40.269 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:41.884 INFO:tasks.workunit.client.0.smithi129.stderr:mv: cannot move 'rename_test_kjQMDP/file_101' to 'rename_test_kjQMDP/rename/file_101': No space left on device 2024-04-25T00:44:42.372 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:42.372 INFO:tasks.workunit.client.0.smithi129.stderr:+ expect_false make_symlinks 101 2024-04-25T00:44:42.372 INFO:tasks.workunit.client.0.smithi129.stderr:+ set -x 2024-04-25T00:44:42.372 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_symlinks 101 2024-04-25T00:44:42.373 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:42.891 INFO:tasks.workunit.client.0.smithi129.stderr:ln: failed to create symbolic link 'symlink_test_B4rokN/symlink/sym_101': No space left on device 2024-04-25T00:44:43.878 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:43.878 INFO:tasks.workunit.client.0.smithi129.stderr:+ expect_false make_links 101 2024-04-25T00:44:43.878 INFO:tasks.workunit.client.0.smithi129.stderr:+ set -x 2024-04-25T00:44:43.879 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_links 101 2024-04-25T00:44:43.879 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:44.966 INFO:tasks.workunit.client.0.smithi129.stderr:ln: failed to create hard link 'link_test_P43CTS/link/link_101': No space left on device 2024-04-25T00:44:45.092 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245c40d640 1 Processor -- start 2024-04-25T00:44:45.092 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245c40d640 1 -- start start 2024-04-25T00:44:45.095 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245c40d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 0x7f24540ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.095 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245c40d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24540b0590 con 0x7f24540d0700 2024-04-25T00:44:45.095 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245c40d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f245406fc80 con 0x7f245406b8a0 2024-04-25T00:44:45.095 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245c40d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24540cd470 con 0x7f24540ccf20 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f24540d0700 0x7f245406b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:51972/0 (socket says 172.21.15.129:51972) 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 -- 172.21.15.129:0/4141851560 learned_addr learned my addr 172.21.15.129:0/4141851560 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a9aa640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 0x7f24540ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285312965 0 0) 0x7f24540b0590 con 0x7f24540d0700 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2440003530 con 0x7f24540d0700 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2717355878 0 0) 0x7f2440003530 con 0x7f24540d0700 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 msgr2=0x7f24540ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 0x7f24540ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.096 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 >> v1:172.21.15.196:6789/0 conn(0x7f245406b8a0 legacy=0x7f245406dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24540cd750 con 0x7f24540d0700 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f24500016a0 con 0x7f24540d0700 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 >> v1:172.21.15.129:6789/0 conn(0x7f24540d0700 legacy=0x7f245406b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24400039f0 0x7f2440003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 0x7f2440004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2440004d50 0x7f24400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24540cd470 con 0x7f24400039f0 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f244000bb90 con 0x7f2440004d50 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f244000bd90 con 0x7f24540ccf20 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24400039f0 0x7f2440003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.097 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a9aa640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2440004d50 0x7f24400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.098 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 -- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 msgr2=0x7f2440004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:45.098 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24540ccf20 0x7f2440004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.098 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 -- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2440004d50 msgr2=0x7f24400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:45.098 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2440004d50 0x7f24400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.098 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a1a9640 1 -- 172.21.15.129:0/4141851560 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2440003530 con 0x7f24400039f0 2024-04-25T00:44:45.098 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.088+0000 7f245a9aa640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2440004d50 0x7f24400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245a1a9640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24400039f0 0x7f2440003e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f245000c160 tx=0x7f245001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ec8b0deb8917af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450007c70 con 0x7f24400039f0 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24500a9070 con 0x7f24400039f0 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f24591a7640 1 -- 172.21.15.129:0/4141851560 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450005d20 con 0x7f24400039f0 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 -- 172.21.15.129:0/4141851560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24400039f0 msgr2=0x7f2440003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 --2- 172.21.15.129:0/4141851560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24400039f0 0x7f2440003e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f245000c160 tx=0x7f245001cb50 comp rx=0 tx=0).stop 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245a9aa640 1 -- 172.21.15.129:0/4141851560 reap_dead start 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 -- 172.21.15.129:0/4141851560 shutdown_connections 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 -- 172.21.15.129:0/4141851560 >> 172.21.15.129:0/4141851560 conn(0x7f24540b2860 msgr2=0x7f24540b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 -- 172.21.15.129:0/4141851560 shutdown_connections 2024-04-25T00:44:45.101 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 -- 172.21.15.129:0/4141851560 wait complete. 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 Processor -- start 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 -- start start 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24400039f0 0x7f24541559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2454151e00 0x7f245415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.092+0000 7f245c40d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 0x7f245413efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245c40d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f245406f990 con 0x7f245415d170 2024-04-25T00:44:45.102 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245c40d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24540d0ae0 con 0x7f2454151e00 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245c40d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f245415d9b0 con 0x7f24400039f0 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 0x7f245413efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 0x7f245413efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50348/0 (socket says 172.21.15.129:50348) 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 -- 172.21.15.129:0/356788201 learned_addr learned my addr 172.21.15.129:0/356788201 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f24599a8640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2454151e00 0x7f245415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a1a9640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24400039f0 0x7f24541559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 -- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24400039f0 msgr2=0x7f24541559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24400039f0 0x7f24541559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 -- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2454151e00 msgr2=0x7f245415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2454151e00 0x7f245415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 -- 172.21.15.129:0/356788201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2454155ef0 con 0x7f245415d170 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f24599a8640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2454151e00 0x7f245415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a9aa640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 0x7f245413efa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f245406f2f0 tx=0x7f244c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:45.103 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c01c070 con 0x7f245415d170 2024-04-25T00:44:45.104 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c00fe90 con 0x7f245415d170 2024-04-25T00:44:45.104 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c017620 con 0x7f245415d170 2024-04-25T00:44:45.104 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24541528d0 con 0x7f245415d170 2024-04-25T00:44:45.104 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2454154ed0 con 0x7f245415d170 2024-04-25T00:44:45.105 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2454068e70 con 0x7f245415d170 2024-04-25T00:44:45.105 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f244c01f070 con 0x7f245415d170 2024-04-25T00:44:45.105 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f244affd640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f24380678f0 0x7f2438069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:45.105 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f244c085ce0 con 0x7f245415d170 2024-04-25T00:44:45.105 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a1a9640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f24380678f0 0x7f2438069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:45.105 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.096+0000 7f245a1a9640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f24380678f0 0x7f2438069db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24500ab000 tx=0x7f24500a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:45.107 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.100+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f244c012070 con 0x7f245415d170 2024-04-25T00:44:45.267 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.260+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mds_dir_max_entries}] v 0) -- 0x7f24540722b0 con 0x7f245415d170 2024-04-25T00:44:45.271 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.264+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_dir_max_entries}]=0 v11)=0 v11) ==== 117+0+0 (secure 0 0 0) 0x7f244c056770 con 0x7f245415d170 2024-04-25T00:44:45.368 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mds_dir_max_entries}] v 0) -- 0x7f24540cf460 con 0x7f245415d170 2024-04-25T00:44:45.369 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f244affd640 1 -- 172.21.15.129:0/356788201 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_dir_max_entries}]=0 v11)=0 v11) ==== 117+0+0 (secure 0 0 0) 0x7f244c056070 con 0x7f245415d170 2024-04-25T00:44:45.371 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f24380678f0 msgr2=0x7f2438069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.371 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f24380678f0 0x7f2438069db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24500ab000 tx=0x7f24500a7070 comp rx=0 tx=0).stop 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 msgr2=0x7f245413efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 0x7f245413efa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f245406f2f0 tx=0x7f244c00f4a0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 shutdown_connections 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f24380678f0 0x7f2438069db0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245415d170 0x7f245413efa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2454151e00 0x7f245415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 --2- 172.21.15.129:0/356788201 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24400039f0 0x7f24541559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 >> 172.21.15.129:0/356788201 conn(0x7f24540b2860 msgr2=0x7f245406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 shutdown_connections 2024-04-25T00:44:45.372 INFO:tasks.workunit.client.0.smithi129.stderr:2024-04-25T00:44:45.364+0000 7f245c40d640 1 -- 172.21.15.129:0/356788201 wait complete. 2024-04-25T00:44:45.403 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.403 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.403 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00591581 s, 0.0 kB/s 2024-04-25T00:44:45.411 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.411 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.411 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00420519 s, 0.0 kB/s 2024-04-25T00:44:45.420 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.420 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.420 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00411224 s, 0.0 kB/s 2024-04-25T00:44:45.429 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.429 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.430 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400492 s, 0.0 kB/s 2024-04-25T00:44:45.438 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.438 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.438 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401541 s, 0.0 kB/s 2024-04-25T00:44:45.447 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.447 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.447 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377358 s, 0.0 kB/s 2024-04-25T00:44:45.456 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.456 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.456 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410292 s, 0.0 kB/s 2024-04-25T00:44:45.465 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.465 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.465 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00390012 s, 0.0 kB/s 2024-04-25T00:44:45.474 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.474 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.474 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00409532 s, 0.0 kB/s 2024-04-25T00:44:45.483 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.483 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.483 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418237 s, 0.0 kB/s 2024-04-25T00:44:45.492 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.492 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.492 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0043942 s, 0.0 kB/s 2024-04-25T00:44:45.501 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.501 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.502 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406671 s, 0.0 kB/s 2024-04-25T00:44:45.511 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.511 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.511 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00433761 s, 0.0 kB/s 2024-04-25T00:44:45.520 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.521 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.521 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00413493 s, 0.0 kB/s 2024-04-25T00:44:45.530 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.530 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.530 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00428874 s, 0.0 kB/s 2024-04-25T00:44:45.539 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.539 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.539 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0036982 s, 0.0 kB/s 2024-04-25T00:44:45.549 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.549 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.549 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00428984 s, 0.0 kB/s 2024-04-25T00:44:45.557 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.557 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.558 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00368982 s, 0.0 kB/s 2024-04-25T00:44:45.566 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.566 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.566 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.004009 s, 0.0 kB/s 2024-04-25T00:44:45.575 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.575 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.576 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00357181 s, 0.0 kB/s 2024-04-25T00:44:45.585 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.586 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.586 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387366 s, 0.0 kB/s 2024-04-25T00:44:45.593 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.593 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.593 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00370806 s, 0.0 kB/s 2024-04-25T00:44:45.602 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.602 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.602 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00380346 s, 0.0 kB/s 2024-04-25T00:44:45.611 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.611 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.611 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00369317 s, 0.0 kB/s 2024-04-25T00:44:45.620 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.620 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.620 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404045 s, 0.0 kB/s 2024-04-25T00:44:45.629 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.629 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.629 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00365208 s, 0.0 kB/s 2024-04-25T00:44:45.638 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.638 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.638 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394966 s, 0.0 kB/s 2024-04-25T00:44:45.647 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.647 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.647 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0036914 s, 0.0 kB/s 2024-04-25T00:44:45.656 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.656 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.656 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038964 s, 0.0 kB/s 2024-04-25T00:44:45.665 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.665 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.665 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0036473 s, 0.0 kB/s 2024-04-25T00:44:45.674 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.674 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.674 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00355902 s, 0.0 kB/s 2024-04-25T00:44:45.683 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.683 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.683 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0037361 s, 0.0 kB/s 2024-04-25T00:44:45.691 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.691 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.691 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00365906 s, 0.0 kB/s 2024-04-25T00:44:45.701 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.701 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.701 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418054 s, 0.0 kB/s 2024-04-25T00:44:45.710 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.710 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.711 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042539 s, 0.0 kB/s 2024-04-25T00:44:45.720 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.720 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.720 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398565 s, 0.0 kB/s 2024-04-25T00:44:45.729 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.729 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.729 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404852 s, 0.0 kB/s 2024-04-25T00:44:45.737 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.738 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.738 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376665 s, 0.0 kB/s 2024-04-25T00:44:45.747 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.747 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.747 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00391721 s, 0.0 kB/s 2024-04-25T00:44:45.757 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.757 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.757 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00455605 s, 0.0 kB/s 2024-04-25T00:44:45.766 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.766 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.766 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377873 s, 0.0 kB/s 2024-04-25T00:44:45.776 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.776 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.776 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00469353 s, 0.0 kB/s 2024-04-25T00:44:45.785 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.785 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.785 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399933 s, 0.0 kB/s 2024-04-25T00:44:45.795 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.795 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.795 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398592 s, 0.0 kB/s 2024-04-25T00:44:45.804 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.804 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.804 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00380806 s, 0.0 kB/s 2024-04-25T00:44:45.814 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.814 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.814 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00463667 s, 0.0 kB/s 2024-04-25T00:44:45.823 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.823 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.823 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00378308 s, 0.0 kB/s 2024-04-25T00:44:45.833 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.833 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.833 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00455275 s, 0.0 kB/s 2024-04-25T00:44:45.842 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.842 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.842 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00384698 s, 0.0 kB/s 2024-04-25T00:44:45.852 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.852 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.852 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423198 s, 0.0 kB/s 2024-04-25T00:44:45.861 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.861 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.861 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379151 s, 0.0 kB/s 2024-04-25T00:44:45.870 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.870 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.870 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00422849 s, 0.0 kB/s 2024-04-25T00:44:45.880 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.880 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.881 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00363235 s, 0.0 kB/s 2024-04-25T00:44:45.890 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.890 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.890 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403436 s, 0.0 kB/s 2024-04-25T00:44:45.899 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.899 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.899 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403068 s, 0.0 kB/s 2024-04-25T00:44:45.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.909 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00464312 s, 0.0 kB/s 2024-04-25T00:44:45.918 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.918 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.918 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00425536 s, 0.0 kB/s 2024-04-25T00:44:45.928 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.928 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.928 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00444073 s, 0.0 kB/s 2024-04-25T00:44:45.938 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.938 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.938 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00478881 s, 0.0 kB/s 2024-04-25T00:44:45.947 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.947 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.947 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388944 s, 0.0 kB/s 2024-04-25T00:44:45.957 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.957 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.957 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423905 s, 0.0 kB/s 2024-04-25T00:44:45.966 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.967 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.967 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388238 s, 0.0 kB/s 2024-04-25T00:44:45.976 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.976 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.976 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00419214 s, 0.0 kB/s 2024-04-25T00:44:45.985 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.985 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.985 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00371321 s, 0.0 kB/s 2024-04-25T00:44:45.994 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:45.994 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:45.994 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426924 s, 0.0 kB/s 2024-04-25T00:44:46.004 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.004 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.004 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418219 s, 0.0 kB/s 2024-04-25T00:44:46.013 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.013 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.013 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00408567 s, 0.0 kB/s 2024-04-25T00:44:46.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.022 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394942 s, 0.0 kB/s 2024-04-25T00:44:46.031 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.032 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.032 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0040493 s, 0.0 kB/s 2024-04-25T00:44:46.041 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.041 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.041 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407109 s, 0.0 kB/s 2024-04-25T00:44:46.050 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.050 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.051 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042008 s, 0.0 kB/s 2024-04-25T00:44:46.059 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.059 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.060 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381468 s, 0.0 kB/s 2024-04-25T00:44:46.069 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.069 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.069 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00438834 s, 0.0 kB/s 2024-04-25T00:44:46.078 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.078 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.078 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00374556 s, 0.0 kB/s 2024-04-25T00:44:46.087 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.087 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.087 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00367647 s, 0.0 kB/s 2024-04-25T00:44:46.096 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.096 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.096 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00413975 s, 0.0 kB/s 2024-04-25T00:44:46.106 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.106 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.106 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426407 s, 0.0 kB/s 2024-04-25T00:44:46.115 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.115 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.115 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381813 s, 0.0 kB/s 2024-04-25T00:44:46.125 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.125 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.125 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00424074 s, 0.0 kB/s 2024-04-25T00:44:46.134 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.134 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.134 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00386354 s, 0.0 kB/s 2024-04-25T00:44:46.143 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.143 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.143 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00415015 s, 0.0 kB/s 2024-04-25T00:44:46.153 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.153 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.153 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00412556 s, 0.0 kB/s 2024-04-25T00:44:46.163 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.163 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.163 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042709 s, 0.0 kB/s 2024-04-25T00:44:46.172 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.172 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.172 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393165 s, 0.0 kB/s 2024-04-25T00:44:46.181 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.182 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.182 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397431 s, 0.0 kB/s 2024-04-25T00:44:46.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.191 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410372 s, 0.0 kB/s 2024-04-25T00:44:46.201 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.201 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.201 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00402457 s, 0.0 kB/s 2024-04-25T00:44:46.209 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.209 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.209 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00441291 s, 0.0 kB/s 2024-04-25T00:44:46.218 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.218 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.218 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381267 s, 0.0 kB/s 2024-04-25T00:44:46.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.228 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00411764 s, 0.0 kB/s 2024-04-25T00:44:46.237 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.237 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.237 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00402686 s, 0.0 kB/s 2024-04-25T00:44:46.246 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.246 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.246 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405613 s, 0.0 kB/s 2024-04-25T00:44:46.256 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.256 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.256 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00409732 s, 0.0 kB/s 2024-04-25T00:44:46.266 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.266 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.266 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00428479 s, 0.0 kB/s 2024-04-25T00:44:46.275 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.275 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.275 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398484 s, 0.0 kB/s 2024-04-25T00:44:46.285 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.285 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.285 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00435978 s, 0.0 kB/s 2024-04-25T00:44:46.294 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.294 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.295 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00424423 s, 0.0 kB/s 2024-04-25T00:44:46.304 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.304 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.304 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00436207 s, 0.0 kB/s 2024-04-25T00:44:46.313 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.313 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.314 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423836 s, 0.0 kB/s 2024-04-25T00:44:46.323 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.323 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.323 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00422079 s, 0.0 kB/s 2024-04-25T00:44:46.332 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.332 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.332 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426839 s, 0.0 kB/s 2024-04-25T00:44:46.343 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.343 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.343 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00454 s, 0.0 kB/s 2024-04-25T00:44:46.352 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.353 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.353 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397636 s, 0.0 kB/s 2024-04-25T00:44:46.362 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.362 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.362 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00439431 s, 0.0 kB/s 2024-04-25T00:44:46.371 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.371 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.371 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00395242 s, 0.0 kB/s 2024-04-25T00:44:46.381 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.381 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.381 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0039959 s, 0.0 kB/s 2024-04-25T00:44:46.390 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.390 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.390 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00364146 s, 0.0 kB/s 2024-04-25T00:44:46.401 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.401 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.401 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00507307 s, 0.0 kB/s 2024-04-25T00:44:46.411 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.411 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.411 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00463433 s, 0.0 kB/s 2024-04-25T00:44:46.420 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.420 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.420 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00386005 s, 0.0 kB/s 2024-04-25T00:44:46.429 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.429 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.429 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00429071 s, 0.0 kB/s 2024-04-25T00:44:46.439 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.439 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.439 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401981 s, 0.0 kB/s 2024-04-25T00:44:46.448 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.448 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.448 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403201 s, 0.0 kB/s 2024-04-25T00:44:46.458 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.458 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.458 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00419744 s, 0.0 kB/s 2024-04-25T00:44:46.467 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.467 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.467 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0041316 s, 0.0 kB/s 2024-04-25T00:44:46.476 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.476 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.476 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398075 s, 0.0 kB/s 2024-04-25T00:44:46.485 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.485 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.486 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403337 s, 0.0 kB/s 2024-04-25T00:44:46.494 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.495 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.495 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376999 s, 0.0 kB/s 2024-04-25T00:44:46.504 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.504 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.504 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00416755 s, 0.0 kB/s 2024-04-25T00:44:46.513 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.513 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.513 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00380583 s, 0.0 kB/s 2024-04-25T00:44:46.522 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.522 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.522 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00414471 s, 0.0 kB/s 2024-04-25T00:44:46.532 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.532 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.532 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00384012 s, 0.0 kB/s 2024-04-25T00:44:46.541 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.541 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.541 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0041175 s, 0.0 kB/s 2024-04-25T00:44:46.550 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.550 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.550 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389352 s, 0.0 kB/s 2024-04-25T00:44:46.559 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.559 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.559 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00385573 s, 0.0 kB/s 2024-04-25T00:44:46.568 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.569 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.569 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00386245 s, 0.0 kB/s 2024-04-25T00:44:46.578 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.578 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.578 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00402617 s, 0.0 kB/s 2024-04-25T00:44:46.586 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.586 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.587 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00366199 s, 0.0 kB/s 2024-04-25T00:44:46.596 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.596 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.596 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00385422 s, 0.0 kB/s 2024-04-25T00:44:46.605 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.605 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.605 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405401 s, 0.0 kB/s 2024-04-25T00:44:46.614 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.614 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.614 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00408025 s, 0.0 kB/s 2024-04-25T00:44:46.623 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.623 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.623 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406423 s, 0.0 kB/s 2024-04-25T00:44:46.632 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.632 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.632 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401228 s, 0.0 kB/s 2024-04-25T00:44:46.642 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.642 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.642 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423995 s, 0.0 kB/s 2024-04-25T00:44:46.651 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.651 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.651 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389934 s, 0.0 kB/s 2024-04-25T00:44:46.661 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.661 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.661 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00430505 s, 0.0 kB/s 2024-04-25T00:44:46.670 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.670 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.670 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404472 s, 0.0 kB/s 2024-04-25T00:44:46.679 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.680 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.680 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00442784 s, 0.0 kB/s 2024-04-25T00:44:46.689 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.689 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.689 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421902 s, 0.0 kB/s 2024-04-25T00:44:46.699 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.699 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.699 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00425218 s, 0.0 kB/s 2024-04-25T00:44:46.708 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.708 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.709 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00446123 s, 0.0 kB/s 2024-04-25T00:44:46.718 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.718 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.718 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00413761 s, 0.0 kB/s 2024-04-25T00:44:46.727 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.727 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.727 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00378561 s, 0.0 kB/s 2024-04-25T00:44:46.735 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.736 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.736 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038485 s, 0.0 kB/s 2024-04-25T00:44:46.744 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.745 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.745 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00392382 s, 0.0 kB/s 2024-04-25T00:44:46.754 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.754 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.754 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407789 s, 0.0 kB/s 2024-04-25T00:44:46.763 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.763 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.763 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00380911 s, 0.0 kB/s 2024-04-25T00:44:46.772 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.772 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.772 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388676 s, 0.0 kB/s 2024-04-25T00:44:46.781 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.781 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.781 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00370628 s, 0.0 kB/s 2024-04-25T00:44:46.790 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.790 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.790 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00396467 s, 0.0 kB/s 2024-04-25T00:44:46.799 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.799 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.799 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00358801 s, 0.0 kB/s 2024-04-25T00:44:46.808 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.808 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.808 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400091 s, 0.0 kB/s 2024-04-25T00:44:46.817 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.817 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.817 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376432 s, 0.0 kB/s 2024-04-25T00:44:46.826 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.826 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.826 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00419826 s, 0.0 kB/s 2024-04-25T00:44:46.835 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.835 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.835 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00357864 s, 0.0 kB/s 2024-04-25T00:44:46.844 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.844 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.844 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00386188 s, 0.0 kB/s 2024-04-25T00:44:46.853 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.853 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.854 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407807 s, 0.0 kB/s 2024-04-25T00:44:46.862 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.862 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.862 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376353 s, 0.0 kB/s 2024-04-25T00:44:46.871 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.872 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.872 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400979 s, 0.0 kB/s 2024-04-25T00:44:46.881 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.881 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.881 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00390963 s, 0.0 kB/s 2024-04-25T00:44:46.890 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.890 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.890 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400104 s, 0.0 kB/s 2024-04-25T00:44:46.899 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.899 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.900 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00386093 s, 0.0 kB/s 2024-04-25T00:44:46.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.909 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389285 s, 0.0 kB/s 2024-04-25T00:44:46.918 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.918 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.918 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383021 s, 0.0 kB/s 2024-04-25T00:44:46.927 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.927 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.927 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00414573 s, 0.0 kB/s 2024-04-25T00:44:46.936 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.936 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.936 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00374104 s, 0.0 kB/s 2024-04-25T00:44:46.945 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.945 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.945 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403701 s, 0.0 kB/s 2024-04-25T00:44:46.954 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.954 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.954 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0039697 s, 0.0 kB/s 2024-04-25T00:44:46.963 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.963 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.963 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00392082 s, 0.0 kB/s 2024-04-25T00:44:46.972 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.972 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.972 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038454 s, 0.0 kB/s 2024-04-25T00:44:46.982 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.982 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.982 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042904 s, 0.0 kB/s 2024-04-25T00:44:46.991 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:46.991 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:46.991 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00385415 s, 0.0 kB/s 2024-04-25T00:44:47.000 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.000 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.000 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00417516 s, 0.0 kB/s 2024-04-25T00:44:47.011 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.011 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.011 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00618964 s, 0.0 kB/s 2024-04-25T00:44:47.021 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.022 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421863 s, 0.0 kB/s 2024-04-25T00:44:47.030 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.030 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.030 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388397 s, 0.0 kB/s 2024-04-25T00:44:47.039 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.039 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.039 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00417639 s, 0.0 kB/s 2024-04-25T00:44:47.048 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.048 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.048 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394315 s, 0.0 kB/s 2024-04-25T00:44:47.057 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.057 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.058 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00408364 s, 0.0 kB/s 2024-04-25T00:44:47.066 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.067 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.067 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397587 s, 0.0 kB/s 2024-04-25T00:44:47.076 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.076 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.076 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401877 s, 0.0 kB/s 2024-04-25T00:44:47.085 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.085 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.085 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00369442 s, 0.0 kB/s 2024-04-25T00:44:47.094 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.094 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.094 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00362877 s, 0.0 kB/s 2024-04-25T00:44:47.104 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.104 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.104 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00462126 s, 0.0 kB/s 2024-04-25T00:44:47.114 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.114 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.114 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400332 s, 0.0 kB/s 2024-04-25T00:44:47.123 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.123 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.123 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394426 s, 0.0 kB/s 2024-04-25T00:44:47.132 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.132 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.132 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410518 s, 0.0 kB/s 2024-04-25T00:44:47.141 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.142 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.142 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405491 s, 0.0 kB/s 2024-04-25T00:44:47.150 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.150 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.150 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00435657 s, 0.0 kB/s 2024-04-25T00:44:47.161 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.161 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.161 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00572289 s, 0.0 kB/s 2024-04-25T00:44:47.170 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.170 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.170 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00372478 s, 0.0 kB/s 2024-04-25T00:44:47.179 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.179 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.179 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00434924 s, 0.0 kB/s 2024-04-25T00:44:47.188 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.188 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.189 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00396977 s, 0.0 kB/s 2024-04-25T00:44:47.198 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.198 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.198 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00443399 s, 0.0 kB/s 2024-04-25T00:44:47.207 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.208 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.208 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377191 s, 0.0 kB/s 2024-04-25T00:44:47.217 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.217 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.217 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406497 s, 0.0 kB/s 2024-04-25T00:44:47.226 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.226 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.226 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00438666 s, 0.0 kB/s 2024-04-25T00:44:47.236 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.236 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.236 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00433954 s, 0.0 kB/s 2024-04-25T00:44:47.245 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.245 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.246 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00427971 s, 0.0 kB/s 2024-04-25T00:44:47.255 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.255 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.255 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00457543 s, 0.0 kB/s 2024-04-25T00:44:47.265 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.265 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.265 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418962 s, 0.0 kB/s 2024-04-25T00:44:47.274 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.274 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.274 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00422537 s, 0.0 kB/s 2024-04-25T00:44:47.283 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.283 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.283 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394266 s, 0.0 kB/s 2024-04-25T00:44:47.293 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.293 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.293 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00425216 s, 0.0 kB/s 2024-04-25T00:44:47.302 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.302 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.302 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038291 s, 0.0 kB/s 2024-04-25T00:44:47.311 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.312 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.312 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0044715 s, 0.0 kB/s 2024-04-25T00:44:47.321 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.321 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.321 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00395205 s, 0.0 kB/s 2024-04-25T00:44:47.330 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.330 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.330 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423447 s, 0.0 kB/s 2024-04-25T00:44:47.339 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.339 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.339 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393559 s, 0.0 kB/s 2024-04-25T00:44:47.349 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.349 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.349 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00424886 s, 0.0 kB/s 2024-04-25T00:44:47.358 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.358 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.358 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398462 s, 0.0 kB/s 2024-04-25T00:44:47.367 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.367 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.368 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00402945 s, 0.0 kB/s 2024-04-25T00:44:47.377 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.377 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.377 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404446 s, 0.0 kB/s 2024-04-25T00:44:47.386 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.386 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.387 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418261 s, 0.0 kB/s 2024-04-25T00:44:47.396 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.396 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.396 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00396362 s, 0.0 kB/s 2024-04-25T00:44:47.406 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.406 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.406 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00470218 s, 0.0 kB/s 2024-04-25T00:44:47.415 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.415 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.415 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00396989 s, 0.0 kB/s 2024-04-25T00:44:47.424 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.424 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.424 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388864 s, 0.0 kB/s 2024-04-25T00:44:47.434 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.434 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.434 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421852 s, 0.0 kB/s 2024-04-25T00:44:47.443 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.443 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.443 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00396263 s, 0.0 kB/s 2024-04-25T00:44:47.453 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.453 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.453 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426722 s, 0.0 kB/s 2024-04-25T00:44:47.462 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.462 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.462 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406437 s, 0.0 kB/s 2024-04-25T00:44:47.471 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.471 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.471 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00419873 s, 0.0 kB/s 2024-04-25T00:44:47.480 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.480 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.481 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387784 s, 0.0 kB/s 2024-04-25T00:44:47.490 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.490 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.490 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406729 s, 0.0 kB/s 2024-04-25T00:44:47.499 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.499 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.499 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00440173 s, 0.0 kB/s 2024-04-25T00:44:47.508 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.508 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.509 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403322 s, 0.0 kB/s 2024-04-25T00:44:47.517 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.517 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.517 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00370033 s, 0.0 kB/s 2024-04-25T00:44:47.526 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.526 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.526 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404342 s, 0.0 kB/s 2024-04-25T00:44:47.535 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.536 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.536 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387256 s, 0.0 kB/s 2024-04-25T00:44:47.545 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.545 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.545 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0041798 s, 0.0 kB/s 2024-04-25T00:44:47.554 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.554 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.554 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381482 s, 0.0 kB/s 2024-04-25T00:44:47.563 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.563 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.563 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410838 s, 0.0 kB/s 2024-04-25T00:44:47.572 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.572 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.573 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00430636 s, 0.0 kB/s 2024-04-25T00:44:47.581 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.582 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.582 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393943 s, 0.0 kB/s 2024-04-25T00:44:47.590 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.590 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.590 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0036838 s, 0.0 kB/s 2024-04-25T00:44:47.599 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.599 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.599 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0039755 s, 0.0 kB/s 2024-04-25T00:44:47.608 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.608 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.608 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00382489 s, 0.0 kB/s 2024-04-25T00:44:47.617 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.617 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.617 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00395997 s, 0.0 kB/s 2024-04-25T00:44:47.627 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.627 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.627 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394236 s, 0.0 kB/s 2024-04-25T00:44:47.636 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.636 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.636 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00411409 s, 0.0 kB/s 2024-04-25T00:44:47.645 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.645 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.645 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00384474 s, 0.0 kB/s 2024-04-25T00:44:47.654 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.654 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.655 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407353 s, 0.0 kB/s 2024-04-25T00:44:47.663 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.664 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.664 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00392781 s, 0.0 kB/s 2024-04-25T00:44:47.672 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.672 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.672 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0037502 s, 0.0 kB/s 2024-04-25T00:44:47.682 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.682 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.682 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00419641 s, 0.0 kB/s 2024-04-25T00:44:47.690 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.691 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.691 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00378505 s, 0.0 kB/s 2024-04-25T00:44:47.700 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.700 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.701 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00448444 s, 0.0 kB/s 2024-04-25T00:44:47.710 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.710 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.710 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397274 s, 0.0 kB/s 2024-04-25T00:44:47.719 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.719 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.719 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401125 s, 0.0 kB/s 2024-04-25T00:44:47.728 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.728 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.728 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00415343 s, 0.0 kB/s 2024-04-25T00:44:47.737 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.738 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.738 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388871 s, 0.0 kB/s 2024-04-25T00:44:47.746 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.746 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.746 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00372158 s, 0.0 kB/s 2024-04-25T00:44:47.756 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.756 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.756 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404925 s, 0.0 kB/s 2024-04-25T00:44:47.765 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.765 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.765 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00391553 s, 0.0 kB/s 2024-04-25T00:44:47.774 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.774 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.774 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389966 s, 0.0 kB/s 2024-04-25T00:44:47.783 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.783 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.783 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00391177 s, 0.0 kB/s 2024-04-25T00:44:47.792 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.792 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.792 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401761 s, 0.0 kB/s 2024-04-25T00:44:47.802 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.802 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.802 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393232 s, 0.0 kB/s 2024-04-25T00:44:47.811 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.811 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.811 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00425403 s, 0.0 kB/s 2024-04-25T00:44:47.820 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.821 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.821 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393191 s, 0.0 kB/s 2024-04-25T00:44:47.830 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.830 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.830 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423544 s, 0.0 kB/s 2024-04-25T00:44:47.839 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.839 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.839 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399933 s, 0.0 kB/s 2024-04-25T00:44:47.851 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.851 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.851 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00686744 s, 0.0 kB/s 2024-04-25T00:44:47.863 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.863 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.863 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0062676 s, 0.0 kB/s 2024-04-25T00:44:47.872 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.872 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.872 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399701 s, 0.0 kB/s 2024-04-25T00:44:47.881 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.881 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.881 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0039701 s, 0.0 kB/s 2024-04-25T00:44:47.890 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.890 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.891 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00416043 s, 0.0 kB/s 2024-04-25T00:44:47.899 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.900 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.900 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393294 s, 0.0 kB/s 2024-04-25T00:44:47.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.909 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042491 s, 0.0 kB/s 2024-04-25T00:44:47.918 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.918 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.918 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405476 s, 0.0 kB/s 2024-04-25T00:44:47.927 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.928 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.928 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407662 s, 0.0 kB/s 2024-04-25T00:44:47.937 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.937 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.937 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407247 s, 0.0 kB/s 2024-04-25T00:44:47.946 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.946 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.946 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403251 s, 0.0 kB/s 2024-04-25T00:44:47.956 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.957 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.957 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00429244 s, 0.0 kB/s 2024-04-25T00:44:47.966 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.966 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.966 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00411367 s, 0.0 kB/s 2024-04-25T00:44:47.975 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.975 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.975 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418941 s, 0.0 kB/s 2024-04-25T00:44:47.984 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.984 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.985 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00409507 s, 0.0 kB/s 2024-04-25T00:44:47.994 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:47.994 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:47.994 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00440334 s, 0.0 kB/s 2024-04-25T00:44:48.003 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.003 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.003 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410551 s, 0.0 kB/s 2024-04-25T00:44:48.013 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.013 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.013 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405511 s, 0.0 kB/s 2024-04-25T00:44:48.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.022 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00397902 s, 0.0 kB/s 2024-04-25T00:44:48.031 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.032 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.032 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00459552 s, 0.0 kB/s 2024-04-25T00:44:48.042 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.043 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.043 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00449463 s, 0.0 kB/s 2024-04-25T00:44:48.054 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.054 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.054 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00471571 s, 0.0 kB/s 2024-04-25T00:44:48.065 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.065 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.065 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00445204 s, 0.0 kB/s 2024-04-25T00:44:48.076 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.077 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.077 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00499506 s, 0.0 kB/s 2024-04-25T00:44:48.087 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.088 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.088 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00424256 s, 0.0 kB/s 2024-04-25T00:44:48.098 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.098 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.099 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00429503 s, 0.0 kB/s 2024-04-25T00:44:48.110 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.110 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.110 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0046333 s, 0.0 kB/s 2024-04-25T00:44:48.120 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.121 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.121 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00396879 s, 0.0 kB/s 2024-04-25T00:44:48.131 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.131 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.131 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00425163 s, 0.0 kB/s 2024-04-25T00:44:48.142 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.142 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.142 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038508 s, 0.0 kB/s 2024-04-25T00:44:48.152 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.152 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.152 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410061 s, 0.0 kB/s 2024-04-25T00:44:48.161 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.161 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.162 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388203 s, 0.0 kB/s 2024-04-25T00:44:48.172 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.172 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.172 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00479313 s, 0.0 kB/s 2024-04-25T00:44:48.181 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.181 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.181 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387444 s, 0.0 kB/s 2024-04-25T00:44:48.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.191 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00382014 s, 0.0 kB/s 2024-04-25T00:44:48.199 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.200 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.200 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407888 s, 0.0 kB/s 2024-04-25T00:44:48.209 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.209 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.209 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00388883 s, 0.0 kB/s 2024-04-25T00:44:48.218 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.218 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.218 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407768 s, 0.0 kB/s 2024-04-25T00:44:48.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.227 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.227 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00379804 s, 0.0 kB/s 2024-04-25T00:44:48.238 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.239 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.239 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00627279 s, 0.0 kB/s 2024-04-25T00:44:48.247 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.249 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.249 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00377674 s, 0.0 kB/s 2024-04-25T00:44:48.257 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.257 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.257 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00422815 s, 0.0 kB/s 2024-04-25T00:44:48.266 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.266 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.266 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387484 s, 0.0 kB/s 2024-04-25T00:44:48.275 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.275 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.275 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00404351 s, 0.0 kB/s 2024-04-25T00:44:48.284 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.284 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.284 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00368643 s, 0.0 kB/s 2024-04-25T00:44:48.293 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.293 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.293 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00403323 s, 0.0 kB/s 2024-04-25T00:44:48.302 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.302 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.302 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00376903 s, 0.0 kB/s 2024-04-25T00:44:48.311 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.312 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.312 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00416199 s, 0.0 kB/s 2024-04-25T00:44:48.320 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.321 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.321 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038671 s, 0.0 kB/s 2024-04-25T00:44:48.330 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.330 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.330 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00417318 s, 0.0 kB/s 2024-04-25T00:44:48.340 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.340 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.340 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00387225 s, 0.0 kB/s 2024-04-25T00:44:48.351 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.351 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.351 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398774 s, 0.0 kB/s 2024-04-25T00:44:48.362 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.362 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.362 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418861 s, 0.0 kB/s 2024-04-25T00:44:48.373 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.373 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.373 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383733 s, 0.0 kB/s 2024-04-25T00:44:48.383 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.383 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.383 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00411393 s, 0.0 kB/s 2024-04-25T00:44:48.394 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.394 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.394 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00385709 s, 0.0 kB/s 2024-04-25T00:44:48.405 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.405 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.405 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00444198 s, 0.0 kB/s 2024-04-25T00:44:48.415 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.415 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.415 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393573 s, 0.0 kB/s 2024-04-25T00:44:48.426 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.426 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.426 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00408816 s, 0.0 kB/s 2024-04-25T00:44:48.437 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.437 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.437 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00432313 s, 0.0 kB/s 2024-04-25T00:44:48.449 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.449 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.449 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00416654 s, 0.0 kB/s 2024-04-25T00:44:48.458 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.458 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.458 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393534 s, 0.0 kB/s 2024-04-25T00:44:48.470 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.470 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.470 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00442721 s, 0.0 kB/s 2024-04-25T00:44:48.480 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.480 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.480 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00382762 s, 0.0 kB/s 2024-04-25T00:44:48.491 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.491 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.491 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00449685 s, 0.0 kB/s 2024-04-25T00:44:48.501 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.501 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.501 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394921 s, 0.0 kB/s 2024-04-25T00:44:48.513 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.513 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.513 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0043338 s, 0.0 kB/s 2024-04-25T00:44:48.523 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.523 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.523 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383646 s, 0.0 kB/s 2024-04-25T00:44:48.533 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.533 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.533 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00398228 s, 0.0 kB/s 2024-04-25T00:44:48.544 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.544 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.544 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00384501 s, 0.0 kB/s 2024-04-25T00:44:48.554 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.554 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.554 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00423673 s, 0.0 kB/s 2024-04-25T00:44:48.565 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.566 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.566 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00408018 s, 0.0 kB/s 2024-04-25T00:44:48.576 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.576 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.576 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00428887 s, 0.0 kB/s 2024-04-25T00:44:48.587 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.587 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.587 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394294 s, 0.0 kB/s 2024-04-25T00:44:48.598 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.598 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.598 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00407652 s, 0.0 kB/s 2024-04-25T00:44:48.608 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.608 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.609 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0039736 s, 0.0 kB/s 2024-04-25T00:44:48.619 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.619 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.619 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410127 s, 0.0 kB/s 2024-04-25T00:44:48.630 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.630 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.630 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00438668 s, 0.0 kB/s 2024-04-25T00:44:48.641 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.641 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.641 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399259 s, 0.0 kB/s 2024-04-25T00:44:48.651 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.652 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.652 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00416584 s, 0.0 kB/s 2024-04-25T00:44:48.662 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.662 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.663 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0043588 s, 0.0 kB/s 2024-04-25T00:44:48.673 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.674 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.674 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00460877 s, 0.0 kB/s 2024-04-25T00:44:48.684 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.684 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.684 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00415994 s, 0.0 kB/s 2024-04-25T00:44:48.695 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.696 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.696 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00455077 s, 0.0 kB/s 2024-04-25T00:44:48.706 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.707 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.707 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00436468 s, 0.0 kB/s 2024-04-25T00:44:48.717 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.717 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.717 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00512066 s, 0.0 kB/s 2024-04-25T00:44:48.727 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.728 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.728 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00461969 s, 0.0 kB/s 2024-04-25T00:44:48.737 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.737 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.737 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00448443 s, 0.0 kB/s 2024-04-25T00:44:48.747 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.747 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.747 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00424784 s, 0.0 kB/s 2024-04-25T00:44:48.757 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.757 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.757 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00453738 s, 0.0 kB/s 2024-04-25T00:44:48.767 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.767 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.767 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00446237 s, 0.0 kB/s 2024-04-25T00:44:48.777 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.777 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.777 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00464033 s, 0.0 kB/s 2024-04-25T00:44:48.786 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.786 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.787 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00454772 s, 0.0 kB/s 2024-04-25T00:44:48.796 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.796 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.796 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00416275 s, 0.0 kB/s 2024-04-25T00:44:48.806 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.806 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.806 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00480576 s, 0.0 kB/s 2024-04-25T00:44:48.816 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.816 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.816 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00466234 s, 0.0 kB/s 2024-04-25T00:44:48.826 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.826 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.826 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00469901 s, 0.0 kB/s 2024-04-25T00:44:48.836 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.836 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.836 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00470285 s, 0.0 kB/s 2024-04-25T00:44:48.846 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.846 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.846 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0046332 s, 0.0 kB/s 2024-04-25T00:44:48.856 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.856 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.856 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00466392 s, 0.0 kB/s 2024-04-25T00:44:48.866 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.866 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.866 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00459721 s, 0.0 kB/s 2024-04-25T00:44:48.876 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.876 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.876 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00464285 s, 0.0 kB/s 2024-04-25T00:44:48.885 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.886 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.886 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00449554 s, 0.0 kB/s 2024-04-25T00:44:48.895 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.896 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.896 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00442495 s, 0.0 kB/s 2024-04-25T00:44:48.905 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.905 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.905 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00427356 s, 0.0 kB/s 2024-04-25T00:44:48.915 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.915 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.915 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00457102 s, 0.0 kB/s 2024-04-25T00:44:48.925 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.925 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.925 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421313 s, 0.0 kB/s 2024-04-25T00:44:48.934 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.934 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.934 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042503 s, 0.0 kB/s 2024-04-25T00:44:48.944 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.944 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.944 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00449338 s, 0.0 kB/s 2024-04-25T00:44:48.954 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.955 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.955 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00484176 s, 0.0 kB/s 2024-04-25T00:44:48.965 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.965 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.965 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00471241 s, 0.0 kB/s 2024-04-25T00:44:48.975 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.975 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.976 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00439468 s, 0.0 kB/s 2024-04-25T00:44:48.985 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.986 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.986 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00477125 s, 0.0 kB/s 2024-04-25T00:44:48.995 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:48.995 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:48.995 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00449203 s, 0.0 kB/s 2024-04-25T00:44:49.006 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.006 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.006 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00512466 s, 0.0 kB/s 2024-04-25T00:44:49.016 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.016 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.016 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00485686 s, 0.0 kB/s 2024-04-25T00:44:49.025 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.026 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.026 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00461705 s, 0.0 kB/s 2024-04-25T00:44:49.035 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.035 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.035 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00441642 s, 0.0 kB/s 2024-04-25T00:44:49.045 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.045 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.045 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00447838 s, 0.0 kB/s 2024-04-25T00:44:49.055 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.055 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.055 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00449706 s, 0.0 kB/s 2024-04-25T00:44:49.065 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.066 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.066 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421393 s, 0.0 kB/s 2024-04-25T00:44:49.074 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.074 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.074 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393211 s, 0.0 kB/s 2024-04-25T00:44:49.083 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.083 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.083 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426165 s, 0.0 kB/s 2024-04-25T00:44:49.093 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.093 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.093 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421051 s, 0.0 kB/s 2024-04-25T00:44:49.103 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.103 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.103 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00507261 s, 0.0 kB/s 2024-04-25T00:44:49.114 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.114 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.114 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00511858 s, 0.0 kB/s 2024-04-25T00:44:49.124 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.124 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.124 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00463431 s, 0.0 kB/s 2024-04-25T00:44:49.133 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.134 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.134 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00437623 s, 0.0 kB/s 2024-04-25T00:44:49.143 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.143 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.143 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00466305 s, 0.0 kB/s 2024-04-25T00:44:49.153 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.153 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.153 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00431664 s, 0.0 kB/s 2024-04-25T00:44:49.163 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.163 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.163 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00470418 s, 0.0 kB/s 2024-04-25T00:44:49.175 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.175 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.175 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00549599 s, 0.0 kB/s 2024-04-25T00:44:49.185 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.185 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.185 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00484848 s, 0.0 kB/s 2024-04-25T00:44:49.195 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.195 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.195 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00487286 s, 0.0 kB/s 2024-04-25T00:44:49.205 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.205 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.205 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00481806 s, 0.0 kB/s 2024-04-25T00:44:49.215 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.215 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.215 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00467157 s, 0.0 kB/s 2024-04-25T00:44:49.226 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.226 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.226 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00494126 s, 0.0 kB/s 2024-04-25T00:44:49.236 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.236 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.236 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00482132 s, 0.0 kB/s 2024-04-25T00:44:49.245 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.245 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.246 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00432978 s, 0.0 kB/s 2024-04-25T00:44:49.255 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.256 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.256 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00484151 s, 0.0 kB/s 2024-04-25T00:44:49.265 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.266 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.266 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00462309 s, 0.0 kB/s 2024-04-25T00:44:49.276 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.276 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.276 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00471233 s, 0.0 kB/s 2024-04-25T00:44:49.285 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.286 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.286 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00452392 s, 0.0 kB/s 2024-04-25T00:44:49.296 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.296 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.296 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00475371 s, 0.0 kB/s 2024-04-25T00:44:49.306 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.306 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.306 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00445009 s, 0.0 kB/s 2024-04-25T00:44:49.316 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.316 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.316 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0049229 s, 0.0 kB/s 2024-04-25T00:44:49.326 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.326 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.326 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00487766 s, 0.0 kB/s 2024-04-25T00:44:49.336 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.336 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.336 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00477396 s, 0.0 kB/s 2024-04-25T00:44:49.346 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.346 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.346 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00438686 s, 0.0 kB/s 2024-04-25T00:44:49.356 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.356 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.356 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0046104 s, 0.0 kB/s 2024-04-25T00:44:49.365 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.365 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.365 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00472155 s, 0.0 kB/s 2024-04-25T00:44:49.375 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.375 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.375 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00456586 s, 0.0 kB/s 2024-04-25T00:44:49.385 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.385 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.386 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00489637 s, 0.0 kB/s 2024-04-25T00:44:49.395 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.395 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.396 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00455081 s, 0.0 kB/s 2024-04-25T00:44:49.406 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.406 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.406 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00512289 s, 0.0 kB/s 2024-04-25T00:44:49.416 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.416 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.416 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00456146 s, 0.0 kB/s 2024-04-25T00:44:49.426 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.426 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.426 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00488517 s, 0.0 kB/s 2024-04-25T00:44:49.436 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.436 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.436 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00441 s, 0.0 kB/s 2024-04-25T00:44:49.445 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.445 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.445 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00468476 s, 0.0 kB/s 2024-04-25T00:44:49.455 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.455 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.455 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00479478 s, 0.0 kB/s 2024-04-25T00:44:49.466 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.466 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.466 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00497926 s, 0.0 kB/s 2024-04-25T00:44:49.476 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.476 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.476 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00455045 s, 0.0 kB/s 2024-04-25T00:44:49.488 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.488 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.488 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00710338 s, 0.0 kB/s 2024-04-25T00:44:49.500 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.501 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.501 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00739727 s, 0.0 kB/s 2024-04-25T00:44:49.511 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.511 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.511 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00560053 s, 0.0 kB/s 2024-04-25T00:44:49.522 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.522 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.522 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00496571 s, 0.0 kB/s 2024-04-25T00:44:49.532 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.532 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.532 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00486055 s, 0.0 kB/s 2024-04-25T00:44:49.542 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.543 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.543 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.005409 s, 0.0 kB/s 2024-04-25T00:44:49.553 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.553 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.554 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00524422 s, 0.0 kB/s 2024-04-25T00:44:49.563 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.564 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.564 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00469187 s, 0.0 kB/s 2024-04-25T00:44:49.575 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.575 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.575 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00533678 s, 0.0 kB/s 2024-04-25T00:44:49.585 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.585 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.585 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00481356 s, 0.0 kB/s 2024-04-25T00:44:49.595 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.595 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.595 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00484889 s, 0.0 kB/s 2024-04-25T00:44:49.605 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.605 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.605 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00472658 s, 0.0 kB/s 2024-04-25T00:44:49.615 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.615 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.616 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00463125 s, 0.0 kB/s 2024-04-25T00:44:49.626 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.626 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.626 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00470181 s, 0.0 kB/s 2024-04-25T00:44:49.636 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.636 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.636 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0045141 s, 0.0 kB/s 2024-04-25T00:44:49.646 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.646 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.646 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0047412 s, 0.0 kB/s 2024-04-25T00:44:49.655 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.655 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.655 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00419866 s, 0.0 kB/s 2024-04-25T00:44:49.665 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.666 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.666 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400974 s, 0.0 kB/s 2024-04-25T00:44:49.674 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.674 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.674 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00394922 s, 0.0 kB/s 2024-04-25T00:44:49.685 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.685 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.685 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00476092 s, 0.0 kB/s 2024-04-25T00:44:49.694 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.694 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.694 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00444099 s, 0.0 kB/s 2024-04-25T00:44:49.704 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.704 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.704 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00456688 s, 0.0 kB/s 2024-04-25T00:44:49.713 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.714 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.714 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00439616 s, 0.0 kB/s 2024-04-25T00:44:49.723 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.723 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.723 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00446191 s, 0.0 kB/s 2024-04-25T00:44:49.733 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.733 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.733 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00410503 s, 0.0 kB/s 2024-04-25T00:44:49.742 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.742 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.742 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00414483 s, 0.0 kB/s 2024-04-25T00:44:49.751 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.751 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.751 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0039962 s, 0.0 kB/s 2024-04-25T00:44:49.761 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.761 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.761 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00425873 s, 0.0 kB/s 2024-04-25T00:44:49.770 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.770 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.770 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389286 s, 0.0 kB/s 2024-04-25T00:44:49.779 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.779 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.779 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00426864 s, 0.0 kB/s 2024-04-25T00:44:49.789 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.789 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.789 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393064 s, 0.0 kB/s 2024-04-25T00:44:49.798 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.798 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.798 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00418834 s, 0.0 kB/s 2024-04-25T00:44:49.807 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.807 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.807 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405956 s, 0.0 kB/s 2024-04-25T00:44:49.817 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.817 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.817 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0042535 s, 0.0 kB/s 2024-04-25T00:44:49.826 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.826 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.826 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00381227 s, 0.0 kB/s 2024-04-25T00:44:49.835 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.835 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.835 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00415592 s, 0.0 kB/s 2024-04-25T00:44:49.845 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.845 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.845 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00389283 s, 0.0 kB/s 2024-04-25T00:44:49.854 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.854 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.854 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0040815 s, 0.0 kB/s 2024-04-25T00:44:49.863 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.863 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.863 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00373309 s, 0.0 kB/s 2024-04-25T00:44:49.873 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.873 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.873 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00421488 s, 0.0 kB/s 2024-04-25T00:44:49.882 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.882 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.882 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0037133 s, 0.0 kB/s 2024-04-25T00:44:49.891 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.891 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.891 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00391874 s, 0.0 kB/s 2024-04-25T00:44:49.900 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.900 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.900 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00382818 s, 0.0 kB/s 2024-04-25T00:44:49.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.909 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.909 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00375903 s, 0.0 kB/s 2024-04-25T00:44:49.919 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.919 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.919 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00405296 s, 0.0 kB/s 2024-04-25T00:44:49.928 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.928 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.928 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00365537 s, 0.0 kB/s 2024-04-25T00:44:49.937 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.938 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.938 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406651 s, 0.0 kB/s 2024-04-25T00:44:49.947 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.947 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.947 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00365717 s, 0.0 kB/s 2024-04-25T00:44:49.956 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.956 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.956 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406195 s, 0.0 kB/s 2024-04-25T00:44:49.965 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.965 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.965 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.003667 s, 0.0 kB/s 2024-04-25T00:44:49.974 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.974 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.975 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00406368 s, 0.0 kB/s 2024-04-25T00:44:49.983 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.983 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.983 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00364741 s, 0.0 kB/s 2024-04-25T00:44:49.992 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:49.993 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:49.993 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00401917 s, 0.0 kB/s 2024-04-25T00:44:50.001 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.002 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.002 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00409915 s, 0.0 kB/s 2024-04-25T00:44:50.011 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.012 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.012 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00413696 s, 0.0 kB/s 2024-04-25T00:44:50.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.022 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.022 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00383512 s, 0.0 kB/s 2024-04-25T00:44:50.032 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.032 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.033 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00392527 s, 0.0 kB/s 2024-04-25T00:44:50.043 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.043 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.043 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0038098 s, 0.0 kB/s 2024-04-25T00:44:50.053 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.054 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.054 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00400521 s, 0.0 kB/s 2024-04-25T00:44:50.064 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.064 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.064 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00390533 s, 0.0 kB/s 2024-04-25T00:44:50.075 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.076 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.076 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00409273 s, 0.0 kB/s 2024-04-25T00:44:50.086 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.086 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.086 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00371225 s, 0.0 kB/s 2024-04-25T00:44:50.096 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.096 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.096 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00399867 s, 0.0 kB/s 2024-04-25T00:44:50.107 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.107 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.107 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00347875 s, 0.0 kB/s 2024-04-25T00:44:50.118 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.118 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.118 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00355517 s, 0.0 kB/s 2024-04-25T00:44:50.128 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.128 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.128 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00375224 s, 0.0 kB/s 2024-04-25T00:44:50.139 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.139 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.139 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00424648 s, 0.0 kB/s 2024-04-25T00:44:50.149 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.150 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.150 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00371702 s, 0.0 kB/s 2024-04-25T00:44:50.160 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.160 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.160 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00402815 s, 0.0 kB/s 2024-04-25T00:44:50.171 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.171 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.171 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00414971 s, 0.0 kB/s 2024-04-25T00:44:50.182 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.182 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.182 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.0049913 s, 0.0 kB/s 2024-04-25T00:44:50.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.191 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.191 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00411287 s, 0.0 kB/s 2024-04-25T00:44:50.200 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.200 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.200 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00393982 s, 0.0 kB/s 2024-04-25T00:44:50.208 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.208 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.208 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00338635 s, 0.0 kB/s 2024-04-25T00:44:50.216 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records in 2024-04-25T00:44:50.217 INFO:tasks.workunit.client.0.smithi129.stderr:0+0 records out 2024-04-25T00:44:50.217 INFO:tasks.workunit.client.0.smithi129.stderr:0 bytes copied, 0.00320641 s, 0.0 kB/s 2024-04-25T00:44:50.218 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:50.218 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_dirs 500 2024-04-25T00:44:50.218 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:55.561 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:44:55.561 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_nodes 500 2024-04-25T00:44:55.561 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:44:57.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:44:57.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:45:00.157 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:45:00.157 INFO:tasks.workunit.client.0.smithi129.stderr:+ rename_files 500 2024-04-25T00:45:00.157 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:45:09.103 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:45:09.103 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_symlinks 500 2024-04-25T00:45:09.103 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:45:11.552 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:45:11.553 INFO:tasks.workunit.client.0.smithi129.stderr:+ make_links 500 2024-04-25T00:45:11.553 INFO:tasks.workunit.client.0.smithi129.stderr:+ set +x 2024-04-25T00:45:17.046 INFO:tasks.workunit.client.0.smithi129.stderr:+ return 0 2024-04-25T00:45:17.046 INFO:tasks.workunit.client.0.smithi129.stderr:+ cleanup 2024-04-25T00:45:17.052 INFO:tasks.workunit.client.0.smithi129.stderr:+ rm -rf link_test_lmebmq link_test_P43CTS link_test_zrQ3lL mkdir_test_CjcUJp mkdir_test_hvJWb8 mkdir_test_nYz8lE mkfile_test_8q3TM3 mkfile_test_Ajddoz mkfile_test_zfUkT1 mknod_test_MTXXHP mknod_test_N1PKXt mknod_test_SSBadR rename_test_fk2IFh rename_test_kjQMDP rename_test_LZAk2f symlink_test_B4rokN symlink_test_le118S symlink_test_soBAQw 2024-04-25T00:45:28.101 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:45:28.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:45:47.469 INFO:tasks.workunit.client.0.smithi129.stderr:+ popd 2024-04-25T00:45:47.474 INFO:tasks.workunit.client.0.smithi129.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-25T00:45:47.474 INFO:tasks.workunit.client.0.smithi129.stdout:OK 2024-04-25T00:45:47.475 INFO:tasks.workunit.client.0.smithi129.stderr:+ echo OK 2024-04-25T00:45:47.475 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T00:45:47.475 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:45:47.533 INFO:tasks.workunit:Stopping ['fs/maxentries'] on client.0... 2024-04-25T00:45:47.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T00:45:47.850 DEBUG:teuthology.parallel:result is None 2024-04-25T00:45:47.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:45:47.881 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:45:47.881 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T00:45:47.893 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T00:45:47.894 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:45:47.894 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:47.932 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:45:47.933 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:47.980 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:45:48.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d40e6b640 1 Processor -- start 2024-04-25T00:45:48.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d40e6b640 1 -- start start 2024-04-25T00:45:48.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d40e6b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7d3c0cfb70 0x7f7d3c165ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d40e6b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c0c6ee0 con 0x7f7d3c0b1510 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d40e6b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c0c71c0 con 0x7f7d3c0b1130 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d40e6b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c0c74a0 con 0x7f7d3c0cfb70 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d39d9b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f7d3c0b1510 0x7f7d3c15ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35098/0 (socket says 172.21.15.129:35098) 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d39d9b640 1 -- 172.21.15.129:0/1058875075 learned_addr learned my addr 172.21.15.129:0/1058875075 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315799738 0 0) 0x7f7d3c0c6ee0 con 0x7f7d3c0b1510 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d24003530 con 0x7f7d3c0b1510 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/1058875075 >> v2:172.21.15.196:3300/0 conn(0x7f7d3c0cfb70 0x7f7d3c165ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3970054351 0 0) 0x7f7d24003530 con 0x7f7d3c0b1510 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 >> v2:172.21.15.196:3300/0 conn(0x7f7d3c0cfb70 msgr2=0x7f7d3c165ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 --2- 172.21.15.129:0/1058875075 >> v2:172.21.15.196:3300/0 conn(0x7f7d3c0cfb70 0x7f7d3c165ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 >> v1:172.21.15.196:6789/0 conn(0x7f7d3c0b1130 legacy=0x7f7d3c15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0c7780 con 0x7f7d3c0b1510 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f7d30001680 con 0x7f7d3c0b1510 2024-04-25T00:45:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 >> v1:172.21.15.129:6789/0 conn(0x7f7d3c0b1510 legacy=0x7f7d3c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d24003a10 0x7f7d24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24004810 0x7f7d24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d240052a0 0x7f7d2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c74a0 con 0x7f7d240052a0 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2400c1f0 con 0x7f7d24004810 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2400c3f0 con 0x7f7d24003a10 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d240052a0 0x7f7d2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3a59c640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d24003a10 0x7f7d24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d39d9b640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24004810 0x7f7d24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d24003a10 msgr2=0x7f7d24003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d24003a10 0x7f7d24003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24004810 msgr2=0x7f7d24004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24004810 0x7f7d24004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.159+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/1058875075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d24003530 con 0x7f7d240052a0 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/1058875075 reap_dead start 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d240052a0 0x7f7d2400b9d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009ba0 tx=0x7f7d3401f080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4cac59345d04ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34009be0 con 0x7f7d240052a0 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d34003ba0 con 0x7f7d240052a0 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3959a640 1 -- 172.21.15.129:0/1058875075 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34004020 con 0x7f7d240052a0 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- 172.21.15.129:0/1058875075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d240052a0 msgr2=0x7f7d2400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d240052a0 0x7f7d2400b9d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009ba0 tx=0x7f7d3401f080 comp rx=0 tx=0).stop 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- 172.21.15.129:0/1058875075 shutdown_connections 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/1058875075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d240052a0 0x7f7d2400b9d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- 172.21.15.129:0/1058875075 >> 172.21.15.129:0/1058875075 conn(0x7f7d3c0cd5b0 msgr2=0x7f7d3c0cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- 172.21.15.129:0/1058875075 shutdown_connections 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- 172.21.15.129:0/1058875075 wait complete. 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 Processor -- start 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- start start 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d240052a0 0x7f7d3c1ef790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d3c1502c0 0x7f7d3c1f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 0x7f7d3c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c6d10 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c6b90 con 0x7f7d240052a0 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d40e6b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c0671c0 con 0x7f7d3c1502c0 2024-04-25T00:45:48.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 0x7f7d3c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 0x7f7d3c1f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33836/0 (socket says 172.21.15.129:33836) 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.163+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/4027788433 learned_addr learned my addr 172.21.15.129:0/4027788433 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3a59c640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d240052a0 0x7f7d3c1ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d39d9b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d3c1502c0 0x7f7d3c1f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d3c1502c0 msgr2=0x7f7d3c1f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d3c1502c0 0x7f7d3c1f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d240052a0 msgr2=0x7f7d3c1ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d240052a0 0x7f7d3c1ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3ad9d640 1 -- 172.21.15.129:0/4027788433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c1f44d0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3ad9d640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 0x7f7d3c1f3f90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7d34007420 tx=0x7f7d3401be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340039a0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d340b0920 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c15dfb0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340b0cf0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c15e570 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c062e50 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7d340ae3f0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d2affd640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7d1c0678c0 0x7f7d1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f7d34121ee0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3a59c640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7d1c0678c0 0x7f7d1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.167+0000 7f7d3a59c640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7d1c0678c0 0x7f7d1c069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c004d30 tx=0x7f7d2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.171+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d340c0060 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.335+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d3c075320 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.335+0000 7f7d2affd640 1 -- 172.21.15.129:0/4027788433 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f7d34018dc0 con 0x7f7d3c1f3b10 2024-04-25T00:45:48.339 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:45:48.339 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:45:48.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7d1c0678c0 msgr2=0x7f7d1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7d1c0678c0 0x7f7d1c069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c004d30 tx=0x7f7d2c009040 comp rx=0 tx=0).stop 2024-04-25T00:45:48.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 msgr2=0x7f7d3c1f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 0x7f7d3c1f3f90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7d34007420 tx=0x7f7d3401be20 comp rx=0 tx=0).stop 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 shutdown_connections 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7d1c0678c0 0x7f7d1c069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7d3c1502c0 0x7f7d3c1f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d3c1f3b10 0x7f7d3c1f3f90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 --2- 172.21.15.129:0/4027788433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d240052a0 0x7f7d3c1ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 >> 172.21.15.129:0/4027788433 conn(0x7f7d3c0cd5b0 msgr2=0x7f7d3c15bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 shutdown_connections 2024-04-25T00:45:48.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.339+0000 7f7d40e6b640 1 -- 172.21.15.129:0/4027788433 wait complete. 2024-04-25T00:45:48.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:45:48.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:45:48.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.479+0000 7fe05efa1640 1 Processor -- start 2024-04-25T00:45:48.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.479+0000 7fe05efa1640 1 -- start start 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05efa1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe0581428c0 0x7fe058169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05efa1640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0580d2710 con 0x7fe058144570 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05efa1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0580d29c0 con 0x7fe058141010 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05efa1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe0580d2ca0 con 0x7fe0581428c0 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe057fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fe058144570 0x7fe058143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35104/0 (socket says 172.21.15.129:35104) 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe057fff640 1 -- 172.21.15.129:0/3841459431 learned_addr learned my addr 172.21.15.129:0/3841459431 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3841459431 >> v2:172.21.15.196:3300/0 conn(0x7fe0581428c0 0x7fe058169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554951635 0 0) 0x7fe0580d2710 con 0x7fe058144570 2024-04-25T00:45:48.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe040003530 con 0x7fe058144570 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1867252676 0 0) 0x7fe040003530 con 0x7fe058144570 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 >> v2:172.21.15.196:3300/0 conn(0x7fe0581428c0 msgr2=0x7fe058169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 --2- 172.21.15.129:0/3841459431 >> v2:172.21.15.196:3300/0 conn(0x7fe0581428c0 0x7fe058169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 >> v1:172.21.15.196:6789/0 conn(0x7fe058141010 legacy=0x7fe0580d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0580d2f80 con 0x7fe058144570 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fe04c001680 con 0x7fe058144570 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 >> v1:172.21.15.129:6789/0 conn(0x7fe058144570 legacy=0x7fe058143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400039c0 0x7fe040003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe040004810 0x7fe040004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0400051d0 0x7fe04000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0580d2ca0 con 0x7fe0400051d0 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe04000c170 con 0x7fe0400039c0 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe04000c370 con 0x7fe040004810 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0400051d0 0x7fe04000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe057fff640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe040004810 0x7fe040004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05cd3d640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400039c0 0x7fe040003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 -- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe040004810 msgr2=0x7fe040004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe040004810 0x7fe040004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 -- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400039c0 msgr2=0x7fe040003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400039c0 0x7fe040003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.483+0000 7fe05d53e640 1 -- 172.21.15.129:0/3841459431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe040003530 con 0x7fe0400051d0 2024-04-25T00:45:48.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 -- 172.21.15.129:0/3841459431 reap_dead start 2024-04-25T00:45:48.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0400051d0 0x7fe04000b9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe050009fa0 tx=0x7fe05001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed40a046a6da143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500b2070 con 0x7fe0400051d0 2024-04-25T00:45:48.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0500a5c40 con 0x7fe0400051d0 2024-04-25T00:45:48.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe0577fe640 1 -- 172.21.15.129:0/3841459431 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050009bf0 con 0x7fe0400051d0 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- 172.21.15.129:0/3841459431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0400051d0 msgr2=0x7fe04000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0400051d0 0x7fe04000b9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe050009fa0 tx=0x7fe05001cc80 comp rx=0 tx=0).stop 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- 172.21.15.129:0/3841459431 shutdown_connections 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3841459431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0400051d0 0x7fe04000b9d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- 172.21.15.129:0/3841459431 >> 172.21.15.129:0/3841459431 conn(0x7fe058072be0 msgr2=0x7fe0580d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- 172.21.15.129:0/3841459431 shutdown_connections 2024-04-25T00:45:48.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- 172.21.15.129:0/3841459431 wait complete. 2024-04-25T00:45:48.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 Processor -- start 2024-04-25T00:45:48.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- start start 2024-04-25T00:45:48.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400051d0 0x7fe05813e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe05813c7f0 0x7fe0581480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 0x7fe05815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581413f0 con 0x7fe05815c980 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0580d3540 con 0x7fe0400051d0 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05efa1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe058141570 con 0x7fe05813c7f0 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 0x7fe05815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 0x7fe05815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33848/0 (socket says 172.21.15.129:33848) 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 -- 172.21.15.129:0/3229494990 learned_addr learned my addr 172.21.15.129:0/3229494990 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe057fff640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe05813c7f0 0x7fe0581480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05cd3d640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400051d0 0x7fe05813e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 -- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe05813c7f0 msgr2=0x7fe0581480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe05813c7f0 0x7fe0581480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 -- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400051d0 msgr2=0x7fe05813e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400051d0 0x7fe05813e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05d53e640 1 -- 172.21.15.129:0/3229494990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05813cc80 con 0x7fe05815c980 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.487+0000 7fe05cd3d640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400051d0 0x7fe05813e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe05d53e640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 0x7fe05815ce10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe05001cab0 tx=0x7fe050009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500b2040 con 0x7fe05815c980 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0500a5a40 con 0x7fe05815c980 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050009bf0 con 0x7fe05815c980 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05813cf60 con 0x7fe05815c980 2024-04-25T00:45:48.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05813d520 con 0x7fe05815c980 2024-04-25T00:45:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe058062e50 con 0x7fe05815c980 2024-04-25T00:45:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe0500c0020 con 0x7fe05815c980 2024-04-25T00:45:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe0557fa640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe0440678c0 0x7fe044069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fe050120370 con 0x7fe05815c980 2024-04-25T00:45:48.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe05cd3d640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe0440678c0 0x7fe044069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.491+0000 7fe05cd3d640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe0440678c0 0x7fe044069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c10 tx=0x7fe048009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.495+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe0500c7020 con 0x7fe05815c980 2024-04-25T00:45:48.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.655+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe05815e200 con 0x7fe05815c980 2024-04-25T00:45:48.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.655+0000 7fe0557fa640 1 -- 172.21.15.129:0/3229494990 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fe050016990 con 0x7fe05815c980 2024-04-25T00:45:48.660 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe0440678c0 msgr2=0x7fe044069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe0440678c0 0x7fe044069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe048000c10 tx=0x7fe048009040 comp rx=0 tx=0).stop 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 msgr2=0x7fe05815ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 0x7fe05815ce10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe05001cab0 tx=0x7fe050009750 comp rx=0 tx=0).stop 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 shutdown_connections 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe0440678c0 0x7fe044069d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe05813c7f0 0x7fe0581480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05815c980 0x7fe05815ce10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 --2- 172.21.15.129:0/3229494990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0400051d0 0x7fe05813e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 >> 172.21.15.129:0/3229494990 conn(0x7fe058072be0 msgr2=0x7fe058161bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:48.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 shutdown_connections 2024-04-25T00:45:48.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.659+0000 7fe05efa1640 1 -- 172.21.15.129:0/3229494990 wait complete. 2024-04-25T00:45:48.680 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:45:48.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f9435002640 1 Processor -- start 2024-04-25T00:45:48.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f9435002640 1 -- start start 2024-04-25T00:45:48.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f9435002640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f943015f3b0 0x7f943015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f9435002640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94300ced70 con 0x7f943015ea60 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f9435002640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9430160aa0 con 0x7f94300cc790 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f9435002640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f943015ff20 con 0x7f943015f3b0 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f943015ea60 0x7f94300ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35116/0 (socket says 172.21.15.129:35116) 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 -- 172.21.15.129:0/2760040030 learned_addr learned my addr 172.21.15.129:0/2760040030 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942ed9d640 1 --2- 172.21.15.129:0/2760040030 >> v2:172.21.15.196:3300/0 conn(0x7f943015f3b0 0x7f943015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2493444083 0 0) 0x7f94300ced70 con 0x7f943015ea60 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9418003530 con 0x7f943015ea60 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3537234369 0 0) 0x7f9418003530 con 0x7f943015ea60 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 >> v2:172.21.15.196:3300/0 conn(0x7f943015f3b0 msgr2=0x7f943015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 --2- 172.21.15.129:0/2760040030 >> v2:172.21.15.196:3300/0 conn(0x7f943015f3b0 0x7f943015ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 >> v1:172.21.15.196:6789/0 conn(0x7f94300cc790 legacy=0x7f94300cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9430160200 con 0x7f943015ea60 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f9428001a00 con 0x7f943015ea60 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 >> v1:172.21.15.129:6789/0 conn(0x7f943015ea60 legacy=0x7f94300ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9418003a10 0x7f9418003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418004810 0x7f9418004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9418005220 0x7f941800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f943015ff20 con 0x7f9418005220 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f941800c170 con 0x7f9418004810 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f941800c370 con 0x7f9418003a10 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9418005220 0x7f941800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942ed9d640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9418003a10 0x7f9418003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942e59c640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418004810 0x7f9418004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 -- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9418003a10 msgr2=0x7f9418003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9418003a10 0x7f9418003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 -- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418004810 msgr2=0x7f9418004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:48.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418004810 0x7f9418004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.807+0000 7f942f59e640 1 -- 172.21.15.129:0/2760040030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9418003530 con 0x7f9418005220 2024-04-25T00:45:48.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f942f59e640 1 -- 172.21.15.129:0/2760040030 reap_dead start 2024-04-25T00:45:48.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f942f59e640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9418005220 0x7f941800b9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f942800a760 tx=0x7f942801f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b62c95eb79b9945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942800a3b0 con 0x7f9418005220 2024-04-25T00:45:48.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94280aa410 con 0x7f9418005220 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f942dd9b640 1 -- 172.21.15.129:0/2760040030 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94280aa800 con 0x7f9418005220 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- 172.21.15.129:0/2760040030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9418005220 msgr2=0x7f941800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9418005220 0x7f941800b9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f942800a760 tx=0x7f942801f9e0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- 172.21.15.129:0/2760040030 shutdown_connections 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 --2- 172.21.15.129:0/2760040030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9418005220 0x7f941800b9d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- 172.21.15.129:0/2760040030 >> 172.21.15.129:0/2760040030 conn(0x7f9430067730 msgr2=0x7f94300b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- 172.21.15.129:0/2760040030 shutdown_connections 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- 172.21.15.129:0/2760040030 wait complete. 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 Processor -- start 2024-04-25T00:45:48.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- start start 2024-04-25T00:45:48.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418005220 0x7f9430141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9430142e80 0x7f94300c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 0x7f94300c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94300cf900 con 0x7f9430143300 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94301607c0 con 0x7f9418005220 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.811+0000 7f9435002640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f943015fba0 con 0x7f9430142e80 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 0x7f94300c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 0x7f94300c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33870/0 (socket says 172.21.15.129:33870) 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 -- 172.21.15.129:0/1204447707 learned_addr learned my addr 172.21.15.129:0/1204447707 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942ed9d640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418005220 0x7f9430141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942e59c640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9430142e80 0x7f94300c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 -- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9430142e80 msgr2=0x7f94300c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9430142e80 0x7f94300c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 -- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418005220 msgr2=0x7f9430141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418005220 0x7f9430141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 -- 172.21.15.129:0/1204447707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9430141de0 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942f59e640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 0x7f94300c40b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f942801f810 tx=0x7f9418005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942800a3b0 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428001260 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94301420c0 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94280aaae0 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9430142680 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9430062e50 con 0x7f9430143300 2024-04-25T00:45:48.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f94280bf060 con 0x7f9430143300 2024-04-25T00:45:48.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f94177fe640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f941c0678c0 0x7f941c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:48.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942ed9d640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f941c0678c0 0x7f941c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:48.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f9428122930 con 0x7f9430143300 2024-04-25T00:45:48.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.815+0000 7f942ed9d640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f941c0678c0 0x7f941c069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94301f5890 tx=0x7f9420009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:48.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.819+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f94280196f0 con 0x7f9430143300 2024-04-25T00:45:48.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.975+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9430075320 con 0x7f9430143300 2024-04-25T00:45:48.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.975+0000 7f94177fe640 1 -- 172.21.15.129:0/1204447707 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f94280f33c0 con 0x7f9430143300 2024-04-25T00:45:48.979 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:45:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f941c0678c0 msgr2=0x7f941c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f941c0678c0 0x7f941c069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94301f5890 tx=0x7f9420009190 comp rx=0 tx=0).stop 2024-04-25T00:45:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 msgr2=0x7f94300c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:48.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 0x7f94300c40b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f942801f810 tx=0x7f9418005840 comp rx=0 tx=0).stop 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 shutdown_connections 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f941c0678c0 0x7f941c069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9430142e80 0x7f94300c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9430143300 0x7f94300c40b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 --2- 172.21.15.129:0/1204447707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9418005220 0x7f9430141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 >> 172.21.15.129:0/1204447707 conn(0x7f9430067730 msgr2=0x7f943006d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 shutdown_connections 2024-04-25T00:45:48.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:48.979+0000 7f9435002640 1 -- 172.21.15.129:0/1204447707 wait complete. 2024-04-25T00:45:48.996 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:45:49.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.107+0000 7f52a2170640 1 Processor -- start 2024-04-25T00:45:49.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.107+0000 7f52a2170640 1 -- start start 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.107+0000 7f52a2170640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f529c070de0 0x7f529c0c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.107+0000 7f52a2170640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c15c7c0 con 0x7f529c0c3b70 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.107+0000 7f52a2170640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529c15caa0 con 0x7f529c0c46a0 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.107+0000 7f52a2170640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f529c15d0f0 con 0x7f529c070de0 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f529c0c3b70 0x7f529c0c4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35132/0 (socket says 172.21.15.129:35132) 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 -- 172.21.15.129:0/3721370940 learned_addr learned my addr 172.21.15.129:0/3721370940 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 959781414 0 0) 0x7f529c15c7c0 con 0x7f529c0c3b70 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5284003530 con 0x7f529c0c3b70 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529bfff640 1 --2- 172.21.15.129:0/3721370940 >> v2:172.21.15.196:3300/0 conn(0x7f529c070de0 0x7f529c0c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 157185964 0 0) 0x7f5284003530 con 0x7f529c0c3b70 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 >> v2:172.21.15.196:3300/0 conn(0x7f529c070de0 msgr2=0x7f529c0c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 --2- 172.21.15.129:0/3721370940 >> v2:172.21.15.196:3300/0 conn(0x7f529c070de0 0x7f529c0c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 >> v1:172.21.15.196:6789/0 conn(0x7f529c0c46a0 legacy=0x7f529c0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c15cd80 con 0x7f529c0c3b70 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f5290001a00 con 0x7f529c0c3b70 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 >> v1:172.21.15.129:6789/0 conn(0x7f529c0c3b70 legacy=0x7f529c0c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52840039c0 0x7f5284003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5284004810 0x7f5284004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52840051d0 0x7f528400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f529c15d0f0 con 0x7f5284004810 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f528400c120 con 0x7f52840039c0 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f528400c320 con 0x7f52840051d0 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5284004810 0x7f5284004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 -- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52840051d0 msgr2=0x7f528400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52840051d0 0x7f528400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 -- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52840039c0 msgr2=0x7f5284003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529b7fe640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52840039c0 0x7f5284003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52840039c0 0x7f5284003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 -- 172.21.15.129:0/3721370940 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5284003530 con 0x7f5284004810 2024-04-25T00:45:49.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529bfff640 1 -- 172.21.15.129:0/3721370940 reap_dead start 2024-04-25T00:45:49.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529affd640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5284004810 0x7f5284004c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f529000a810 tx=0x7f529001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18a7be16f6f9b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000a460 con 0x7f5284004810 2024-04-25T00:45:49.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52900aa6f0 con 0x7f5284004810 2024-04-25T00:45:49.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f529a7fc640 1 -- 172.21.15.129:0/3721370940 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900aab20 con 0x7f5284004810 2024-04-25T00:45:49.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 -- 172.21.15.129:0/3721370940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5284004810 msgr2=0x7f5284004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5284004810 0x7f5284004c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f529000a810 tx=0x7f529001faa0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 -- 172.21.15.129:0/3721370940 shutdown_connections 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 --2- 172.21.15.129:0/3721370940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5284004810 0x7f5284004c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 -- 172.21.15.129:0/3721370940 >> 172.21.15.129:0/3721370940 conn(0x7f529c0d12c0 msgr2=0x7f529c0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 -- 172.21.15.129:0/3721370940 shutdown_connections 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.111+0000 7f52a2170640 1 -- 172.21.15.129:0/3721370940 wait complete. 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 Processor -- start 2024-04-25T00:45:49.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- start start 2024-04-25T00:45:49.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5284004810 0x7f529c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 0x7f529c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c071680 0x7f529c071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f529c0706d0 con 0x7f529c0cb4d0 2024-04-25T00:45:49.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f529c15c470 con 0x7f5284004810 2024-04-25T00:45:49.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f529c070850 con 0x7f529c071680 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 0x7f529c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 0x7f529c0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33888/0 (socket says 172.21.15.129:33888) 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 -- 172.21.15.129:0/2058088701 learned_addr learned my addr 172.21.15.129:0/2058088701 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529b7fe640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5284004810 0x7f529c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529bfff640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c071680 0x7f529c071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 -- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c071680 msgr2=0x7f529c071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c071680 0x7f529c071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 -- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5284004810 msgr2=0x7f529c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5284004810 0x7f529c14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 -- 172.21.15.129:0/2058088701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c14f240 con 0x7f529c0cb4d0 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529b7fe640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5284004810 0x7f529c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f529affd640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 0x7f529c0cb950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f529001f8d0 tx=0x7f5284005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000a460 con 0x7f529c0cb4d0 2024-04-25T00:45:49.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52900b8070 con 0x7f529c0cb4d0 2024-04-25T00:45:49.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529c0c7040 con 0x7f529c0cb4d0 2024-04-25T00:45:49.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290001fc0 con 0x7f529c0cb4d0 2024-04-25T00:45:49.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529c0c7600 con 0x7f529c0cb4d0 2024-04-25T00:45:49.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.115+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c062e50 con 0x7f529c0cb4d0 2024-04-25T00:45:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.119+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52900ba020 con 0x7f529c0cb4d0 2024-04-25T00:45:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.119+0000 7f5283fff640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f52880678c0 0x7f5288069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.119+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f5290123890 con 0x7f529c0cb4d0 2024-04-25T00:45:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.119+0000 7f529b7fe640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f52880678c0 0x7f5288069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.119+0000 7f529b7fe640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f52880678c0 0x7f5288069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f529c0ca7e0 tx=0x7f528c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.123+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f52900cb080 con 0x7f529c0cb4d0 2024-04-25T00:45:49.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f529c075320 con 0x7f529c0cb4d0 2024-04-25T00:45:49.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f5283fff640 1 -- 172.21.15.129:0/2058088701 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f52900ae030 con 0x7f529c0cb4d0 2024-04-25T00:45:49.325 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:45:49.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f52880678c0 msgr2=0x7f5288069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f52880678c0 0x7f5288069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f529c0ca7e0 tx=0x7f528c009040 comp rx=0 tx=0).stop 2024-04-25T00:45:49.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 msgr2=0x7f529c0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 0x7f529c0cb950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f529001f8d0 tx=0x7f5284005840 comp rx=0 tx=0).stop 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 shutdown_connections 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f52880678c0 0x7f5288069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f529c071680 0x7f529c071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f529c0cb4d0 0x7f529c0cb950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 --2- 172.21.15.129:0/2058088701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5284004810 0x7f529c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 >> 172.21.15.129:0/2058088701 conn(0x7f529c0d12c0 msgr2=0x7f529c072600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.323+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 shutdown_connections 2024-04-25T00:45:49.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.327+0000 7f52a2170640 1 -- 172.21.15.129:0/2058088701 wait complete. 2024-04-25T00:45:49.345 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:45:49.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 Processor -- start 2024-04-25T00:45:49.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 -- start start 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a415bf50 0x7f82a415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f82a40cd260 0x7f82a40aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a406d1b0 con 0x7f82a40aa0c0 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f82a406d490 con 0x7f82a40cd260 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82aafa8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f82a406d770 con 0x7f82a415bf50 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f82a40cd260 0x7f82a40aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f82a40cd260 0x7f82a40aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33898/0 (socket says 172.21.15.129:33898) 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 -- 172.21.15.129:0/2290803882 learned_addr learned my addr 172.21.15.129:0/2290803882 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a8d44640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a415bf50 0x7f82a415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 -- 172.21.15.129:0/2290803882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a415bf50 msgr2=0x7f82a415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a415bf50 0x7f82a415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 -- 172.21.15.129:0/2290803882 >> v1:172.21.15.196:6789/0 conn(0x7f82a40aa0c0 legacy=0x7f82a40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a9545640 1 -- 172.21.15.129:0/2290803882 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a406da50 con 0x7f82a40cd260 2024-04-25T00:45:49.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.459+0000 7f82a8d44640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a415bf50 0x7f82a415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:49.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a9545640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40cd260 0x7f82a40aa520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f829c009e50 tx=0x7f829c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e80ec6c8a58bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a37fe640 1 -- 172.21.15.129:0/2290803882 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0a5860 con 0x7f82a40cd260 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a37fe640 1 -- 172.21.15.129:0/2290803882 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c0a5a00 con 0x7f82a40cd260 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a37fe640 1 -- 172.21.15.129:0/2290803882 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0a5dd0 con 0x7f82a40cd260 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- 172.21.15.129:0/2290803882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40cd260 msgr2=0x7f82a40aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40cd260 0x7f82a40aa520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f829c009e50 tx=0x7f829c01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- 172.21.15.129:0/2290803882 shutdown_connections 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40cd260 0x7f82a40aa520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 --2- 172.21.15.129:0/2290803882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a415bf50 0x7f82a415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- 172.21.15.129:0/2290803882 >> 172.21.15.129:0/2290803882 conn(0x7f82a40b14a0 msgr2=0x7f82a40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- 172.21.15.129:0/2290803882 shutdown_connections 2024-04-25T00:45:49.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- 172.21.15.129:0/2290803882 wait complete. 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 Processor -- start 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- start start 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 0x7f82a4155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a40cd260 0x7f82a41522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a415bf50 0x7f82a4145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82a406ea60 con 0x7f82a40aa0c0 2024-04-25T00:45:49.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82a406cf70 con 0x7f82a415bf50 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82aafa8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f82a406ebe0 con 0x7f82a40cd260 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a8d44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 0x7f82a4155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a8d44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 0x7f82a4155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33902/0 (socket says 172.21.15.129:33902) 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a8d44640 1 -- 172.21.15.129:0/445541790 learned_addr learned my addr 172.21.15.129:0/445541790 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a3fff640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a40cd260 0x7f82a41522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a8d44640 1 -- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a40cd260 msgr2=0x7f82a41522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a8d44640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a40cd260 0x7f82a41522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a8d44640 1 -- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a415bf50 msgr2=0x7f82a4145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.463+0000 7f82a9545640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a415bf50 0x7f82a4145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a8d44640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a415bf50 0x7f82a4145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a8d44640 1 -- 172.21.15.129:0/445541790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a4146130 con 0x7f82a40aa0c0 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a3fff640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a40cd260 0x7f82a41522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:49.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a8d44640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 0x7f82a4155840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f829800c820 tx=0x7f829800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298015070 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8298007a30 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a4147ba0 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298007e40 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a4152dd0 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a4062e50 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8298018070 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a17fa640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f8288067940 0x7f8288069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f8298085b00 con 0x7f82a40aa0c0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a3fff640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f8288067940 0x7f8288069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.467+0000 7f82a3fff640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f8288067940 0x7f8288069e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82a4155d80 tx=0x7f8290009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.471+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8298056670 con 0x7f82a40aa0c0 2024-04-25T00:45:49.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.643+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f82a4075320 con 0x7f82a40aa0c0 2024-04-25T00:45:49.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.643+0000 7f82a17fa640 1 -- 172.21.15.129:0/445541790 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f8298055f70 con 0x7f82a40aa0c0 2024-04-25T00:45:49.646 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f8288067940 msgr2=0x7f8288069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f8288067940 0x7f8288069e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82a4155d80 tx=0x7f8290009040 comp rx=0 tx=0).stop 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 msgr2=0x7f82a4155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 0x7f82a4155840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f829800c820 tx=0x7f829800cce0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 shutdown_connections 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f8288067940 0x7f8288069e00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82a40cd260 0x7f82a41522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82a415bf50 0x7f82a4145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 --2- 172.21.15.129:0/445541790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82a40aa0c0 0x7f82a4155840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 >> 172.21.15.129:0/445541790 conn(0x7f82a40b14a0 msgr2=0x7f82a4159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:49.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 shutdown_connections 2024-04-25T00:45:49.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.647+0000 7f82aafa8640 1 -- 172.21.15.129:0/445541790 wait complete. 2024-04-25T00:45:49.665 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:45:49.665 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:49.671 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:45:49.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 Processor -- start 2024-04-25T00:45:49.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 -- start start 2024-04-25T00:45:49.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe19415e470 0x7fe19406e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1940b0e70 0x7fe194070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19406cf00 con 0x7fe1940cc690 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fe19406d1e0 con 0x7fe1940b0e70 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe199505640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe19406d4c0 con 0x7fe19415e470 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1940b0e70 0x7fe194070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe1940b0e70 0x7fe194070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33912/0 (socket says 172.21.15.129:33912) 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 -- 172.21.15.129:0/1922397672 learned_addr learned my addr 172.21.15.129:0/1922397672 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe192ffd640 1 --2- 172.21.15.129:0/1922397672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe19415e470 0x7fe19406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 -- 172.21.15.129:0/1922397672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe19415e470 msgr2=0x7fe19406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 --2- 172.21.15.129:0/1922397672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe19415e470 0x7fe19406e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 -- 172.21.15.129:0/1922397672 >> v1:172.21.15.196:6789/0 conn(0x7fe1940cc690 legacy=0x7fe1940b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:49.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.827+0000 7fe1937fe640 1 -- 172.21.15.129:0/1922397672 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19406d7a0 con 0x7fe1940b0e70 2024-04-25T00:45:49.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe1937fe640 1 --2- 172.21.15.129:0/1922397672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe194070770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe184009e50 tx=0x7fe18401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533a7f3a7e051261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe191ffb640 1 -- 172.21.15.129:0/1922397672 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840a56f0 con 0x7fe1940b0e70 2024-04-25T00:45:49.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe191ffb640 1 -- 172.21.15.129:0/1922397672 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1840a5890 con 0x7fe1940b0e70 2024-04-25T00:45:49.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe191ffb640 1 -- 172.21.15.129:0/1922397672 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840a5c60 con 0x7fe1940b0e70 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 -- 172.21.15.129:0/1922397672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 msgr2=0x7fe194070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 --2- 172.21.15.129:0/1922397672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe194070770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe184009e50 tx=0x7fe18401c960 comp rx=0 tx=0).stop 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 -- 172.21.15.129:0/1922397672 shutdown_connections 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 --2- 172.21.15.129:0/1922397672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe194070770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 --2- 172.21.15.129:0/1922397672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe19415e470 0x7fe19406e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 -- 172.21.15.129:0/1922397672 >> 172.21.15.129:0/1922397672 conn(0x7fe1940b14a0 msgr2=0x7fe1940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 -- 172.21.15.129:0/1922397672 shutdown_connections 2024-04-25T00:45:49.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 -- 172.21.15.129:0/1922397672 wait complete. 2024-04-25T00:45:49.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 Processor -- start 2024-04-25T00:45:49.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.831+0000 7fe199505640 1 -- start start 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe19414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1940cc690 0x7fe19414d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe19415e470 0x7fe19414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe194071850 con 0x7fe1940b0e70 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1940716d0 con 0x7fe19415e470 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1940719d0 con 0x7fe1940cc690 2024-04-25T00:45:49.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe19414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe19414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33922/0 (socket says 172.21.15.129:33922) 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 -- 172.21.15.129:0/2301513716 learned_addr learned my addr 172.21.15.129:0/2301513716 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe1927fc640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1940cc690 0x7fe19414d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe1937fe640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe19415e470 0x7fe19414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 -- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1940cc690 msgr2=0x7fe19414d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1940cc690 0x7fe19414d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 -- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe19415e470 msgr2=0x7fe19414cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe19415e470 0x7fe19414cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:49.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 -- 172.21.15.129:0/2301513716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19414db60 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe192ffd640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe19414e980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00d800 tx=0x7fe18c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00b6c0 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18c019070 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19414de00 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00ba60 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19414e3c0 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe194062e50 con 0x7fe1940b0e70 2024-04-25T00:45:49.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe18c00bc00 con 0x7fe1940b0e70 2024-04-25T00:45:49.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe17b7fe640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1800678c0 0x7fe180069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:49.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fe18c086190 con 0x7fe1940b0e70 2024-04-25T00:45:49.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe1927fc640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1800678c0 0x7fe180069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:49.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.835+0000 7fe1927fc640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1800678c0 0x7fe180069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe19414f280 tx=0x7fe188008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:49.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:49.839+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe18c056cb0 con 0x7fe1940b0e70 2024-04-25T00:45:50.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe194075320 con 0x7fe1940b0e70 2024-04-25T00:45:50.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe17b7fe640 1 -- 172.21.15.129:0/2301513716 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7fe18c0565b0 con 0x7fe1940b0e70 2024-04-25T00:45:50.005 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:45:50.005 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:45:50.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1800678c0 msgr2=0x7fe180069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1800678c0 0x7fe180069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe19414f280 tx=0x7fe188008040 comp rx=0 tx=0).stop 2024-04-25T00:45:50.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 msgr2=0x7fe19414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe19414e980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00d800 tx=0x7fe18c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 shutdown_connections 2024-04-25T00:45:50.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe1800678c0 0x7fe180069d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1940cc690 0x7fe19414d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe19415e470 0x7fe19414cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 --2- 172.21.15.129:0/2301513716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1940b0e70 0x7fe19414e980 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 >> 172.21.15.129:0/2301513716 conn(0x7fe1940b14a0 msgr2=0x7fe1940adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.003+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 shutdown_connections 2024-04-25T00:45:50.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.007+0000 7fe199505640 1 -- 172.21.15.129:0/2301513716 wait complete. 2024-04-25T00:45:50.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:45:50.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:45:50.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.155+0000 7f0cf3097640 1 Processor -- start 2024-04-25T00:45:50.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.155+0000 7f0cf3097640 1 -- start start 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0cec143e80 0x7f0cec1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec0d2710 con 0x7f0cec141010 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec0d29c0 con 0x7f0cec143e80 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec0d2ca0 con 0x7f0cec144570 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf0e33640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0cec141010 0x7f0cec0d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35134/0 (socket says 172.21.15.129:35134) 2024-04-25T00:45:50.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf0e33640 1 -- 172.21.15.129:0/1895901927 learned_addr learned my addr 172.21.15.129:0/1895901927 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf1634640 1 --2- 172.21.15.129:0/1895901927 >> v2:172.21.15.129:3300/0 conn(0x7f0cec143e80 0x7f0cec1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cebfff640 1 --2- 172.21.15.129:0/1895901927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0ceb7fe640 1 -- 172.21.15.129:0/1895901927 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955483793 0 0) 0x7f0cec0d2710 con 0x7f0cec141010 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0ceb7fe640 1 -- 172.21.15.129:0/1895901927 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd4003530 con 0x7f0cec141010 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf1634640 1 -- 172.21.15.129:0/1895901927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 msgr2=0x7f0cec143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf1634640 1 --2- 172.21.15.129:0/1895901927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf1634640 1 -- 172.21.15.129:0/1895901927 >> v1:172.21.15.129:6789/0 conn(0x7f0cec141010 legacy=0x7f0cec0d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf1634640 1 -- 172.21.15.129:0/1895901927 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec0d2f80 con 0x7f0cec143e80 2024-04-25T00:45:50.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf1634640 1 --2- 172.21.15.129:0/1895901927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1418b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009d90 tx=0x7f0ce401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b544de50a3ae3be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0ceb7fe640 1 -- 172.21.15.129:0/1895901927 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40b2070 con 0x7f0cec143e80 2024-04-25T00:45:50.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0ceb7fe640 1 -- 172.21.15.129:0/1895901927 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce40a5890 con 0x7f0cec143e80 2024-04-25T00:45:50.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0ceb7fe640 1 -- 172.21.15.129:0/1895901927 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40a5c60 con 0x7f0cec143e80 2024-04-25T00:45:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- 172.21.15.129:0/1895901927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 msgr2=0x7f0cec1418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 --2- 172.21.15.129:0/1895901927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1418b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009d90 tx=0x7f0ce401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- 172.21.15.129:0/1895901927 shutdown_connections 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 --2- 172.21.15.129:0/1895901927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1418b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 --2- 172.21.15.129:0/1895901927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- 172.21.15.129:0/1895901927 >> 172.21.15.129:0/1895901927 conn(0x7f0cec072be0 msgr2=0x7f0cec0d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- 172.21.15.129:0/1895901927 shutdown_connections 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.159+0000 7f0cf3097640 1 -- 172.21.15.129:0/1895901927 wait complete. 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 Processor -- start 2024-04-25T00:45:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- start start 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec141010 0x7f0cec1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec15fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec142780 con 0x7f0cec143e80 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec142600 con 0x7f0cec141010 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec142900 con 0x7f0cec144570 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1fab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33930/0 (socket says 172.21.15.129:33930) 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 -- 172.21.15.129:0/143751561 learned_addr learned my addr 172.21.15.129:0/143751561 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf1634640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec15fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf0e33640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec141010 0x7f0cec1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 -- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 msgr2=0x7f0cec15fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec15fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 -- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec141010 msgr2=0x7f0cec1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec141010 0x7f0cec1545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 -- 172.21.15.129:0/143751561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec13def0 con 0x7f0cec143e80 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf1634640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec15fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf0e33640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec141010 0x7f0cec1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cebfff640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1fab50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000e8d0 tx=0x7f0ce000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0017070 con 0x7f0cec143e80 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce000c940 con 0x7f0cec143e80 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec0b2470 con 0x7f0cec143e80 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000ccf0 con 0x7f0cec143e80 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec0b1b40 con 0x7f0cec143e80 2024-04-25T00:45:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.163+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec062e50 con 0x7f0cec143e80 2024-04-25T00:45:50.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.167+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0ce001a070 con 0x7f0cec143e80 2024-04-25T00:45:50.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.167+0000 7f0ce97fa640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0cd80678c0 0x7f0cd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.167+0000 7f0cf0e33640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0cd80678c0 0x7f0cd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.167+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f0ce0085c80 con 0x7f0cec143e80 2024-04-25T00:45:50.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.167+0000 7f0cf0e33640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0cd80678c0 0x7f0cd8069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc005890 tx=0x7f0cdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.171+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0cec13def0 con 0x7f0cec143e80 2024-04-25T00:45:50.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0cec075320 con 0x7f0cec143e80 2024-04-25T00:45:50.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0ce97fa640 1 -- 172.21.15.129:0/143751561 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f0ce00567a0 con 0x7f0cec143e80 2024-04-25T00:45:50.333 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:45:50.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0cd80678c0 msgr2=0x7f0cd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0cd80678c0 0x7f0cd8069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc005890 tx=0x7f0cdc009040 comp rx=0 tx=0).stop 2024-04-25T00:45:50.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 msgr2=0x7f0cec1fab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1fab50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000e8d0 tx=0x7f0ce000ed90 comp rx=0 tx=0).stop 2024-04-25T00:45:50.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 shutdown_connections 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0cd80678c0 0x7f0cd8069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec144570 0x7f0cec15fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec143e80 0x7f0cec1fab50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 --2- 172.21.15.129:0/143751561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec141010 0x7f0cec1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 >> 172.21.15.129:0/143751561 conn(0x7f0cec072be0 msgr2=0x7f0cec15c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 shutdown_connections 2024-04-25T00:45:50.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.331+0000 7f0cf3097640 1 -- 172.21.15.129:0/143751561 wait complete. 2024-04-25T00:45:50.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:45:50.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f874077f640 1 Processor -- start 2024-04-25T00:45:50.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f874077f640 1 -- start start 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f874077f640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f873815eec0 0x7f8738162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f874077f640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873815f450 con 0x7f8738070da0 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f874077f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873815f730 con 0x7f8738141210 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f874077f640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f87380aee10 con 0x7f873815eec0 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f873ed1c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f873815eec0 0x7f8738162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f873dd1a640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f8738070da0 0x7f873815e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35144/0 (socket says 172.21.15.129:35144) 2024-04-25T00:45:50.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f873dd1a640 1 -- 172.21.15.129:0/2869507588 learned_addr learned my addr 172.21.15.129:0/2869507588 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:50.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f873ed1c640 1 -- 172.21.15.129:0/2869507588 >> v1:172.21.15.196:6789/0 conn(0x7f8738141210 legacy=0x7f8738070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f873ed1c640 1 -- 172.21.15.129:0/2869507588 >> v1:172.21.15.129:6789/0 conn(0x7f8738070da0 legacy=0x7f873815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.479+0000 7f873ed1c640 1 -- 172.21.15.129:0/2869507588 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8738161e90 con 0x7f873815eec0 2024-04-25T00:45:50.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873ed1c640 1 --2- 172.21.15.129:0/2869507588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f873815eec0 0x7f8738162970 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8730009d90 tx=0x7f873001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bc24090ed56abc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873d519640 1 -- 172.21.15.129:0/2869507588 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87300b2070 con 0x7f873815eec0 2024-04-25T00:45:50.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873d519640 1 -- 172.21.15.129:0/2869507588 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87300a5890 con 0x7f873815eec0 2024-04-25T00:45:50.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873d519640 1 -- 172.21.15.129:0/2869507588 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87300a5c60 con 0x7f873815eec0 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- 172.21.15.129:0/2869507588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f873815eec0 msgr2=0x7f8738162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 --2- 172.21.15.129:0/2869507588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f873815eec0 0x7f8738162970 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8730009d90 tx=0x7f873001c8e0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- 172.21.15.129:0/2869507588 shutdown_connections 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 --2- 172.21.15.129:0/2869507588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f873815eec0 0x7f8738162970 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- 172.21.15.129:0/2869507588 >> 172.21.15.129:0/2869507588 conn(0x7f8738067730 msgr2=0x7f87380b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- 172.21.15.129:0/2869507588 shutdown_connections 2024-04-25T00:45:50.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- 172.21.15.129:0/2869507588 wait complete. 2024-04-25T00:45:50.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 Processor -- start 2024-04-25T00:45:50.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- start start 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 0x7f8738160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8738141210 0x7f8738153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873815eec0 0x7f8738154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87380cc9a0 con 0x7f8738070da0 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87380cc820 con 0x7f873815eec0 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f874077f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87380ccb20 con 0x7f8738141210 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873e51b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 0x7f8738160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873e51b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 0x7f8738160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33940/0 (socket says 172.21.15.129:33940) 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.483+0000 7f873e51b640 1 -- 172.21.15.129:0/2277668451 learned_addr learned my addr 172.21.15.129:0/2277668451 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:50.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873ed1c640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873815eec0 0x7f8738154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873dd1a640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8738141210 0x7f8738153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873e51b640 1 -- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8738141210 msgr2=0x7f8738153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873e51b640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8738141210 0x7f8738153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873e51b640 1 -- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873815eec0 msgr2=0x7f8738154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873e51b640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873815eec0 0x7f8738154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873e51b640 1 -- 172.21.15.129:0/2277668451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873813e940 con 0x7f8738070da0 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873dd1a640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8738141210 0x7f8738153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873e51b640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 0x7f8738160ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f873400c450 tx=0x7f873400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734016020 con 0x7f8738070da0 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87340052d0 con 0x7f8738070da0 2024-04-25T00:45:50.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873813ec20 con 0x7f8738070da0 2024-04-25T00:45:50.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734007260 con 0x7f8738070da0 2024-04-25T00:45:50.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381611e0 con 0x7f8738070da0 2024-04-25T00:45:50.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8738062e50 con 0x7f8738070da0 2024-04-25T00:45:50.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8734007400 con 0x7f8738070da0 2024-04-25T00:45:50.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f872effd640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f871c0678c0 0x7f871c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f8734085670 con 0x7f8738070da0 2024-04-25T00:45:50.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873dd1a640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f871c0678c0 0x7f871c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.487+0000 7f873dd1a640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f871c0678c0 0x7f871c069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87381fa8b0 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.491+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8734020020 con 0x7f8738070da0 2024-04-25T00:45:50.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.651+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8738075320 con 0x7f8738070da0 2024-04-25T00:45:50.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f872effd640 1 -- 172.21.15.129:0/2277668451 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f8734005480 con 0x7f8738070da0 2024-04-25T00:45:50.656 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f871c0678c0 msgr2=0x7f871c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f871c0678c0 0x7f871c069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f87381fa8b0 tx=0x7f8728009040 comp rx=0 tx=0).stop 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 msgr2=0x7f8738160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 0x7f8738160ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f873400c450 tx=0x7f873400c910 comp rx=0 tx=0).stop 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 shutdown_connections 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f871c0678c0 0x7f871c069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8738141210 0x7f8738153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873815eec0 0x7f8738154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 --2- 172.21.15.129:0/2277668451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8738070da0 0x7f8738160ca0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 >> 172.21.15.129:0/2277668451 conn(0x7f8738067730 msgr2=0x7f8738160230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 shutdown_connections 2024-04-25T00:45:50.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.655+0000 7f874077f640 1 -- 172.21.15.129:0/2277668451 wait complete. 2024-04-25T00:45:50.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:45:50.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 Processor -- start 2024-04-25T00:45:50.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 -- start start 2024-04-25T00:45:50.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4538143b80 0x7f4538144000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f453806c6c0 0x7f453806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f453815ed70 con 0x7f4538141560 2024-04-25T00:45:50.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4538071ed0 con 0x7f4538143b80 2024-04-25T00:45:50.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ecfa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4538144730 con 0x7f453806c6c0 2024-04-25T00:45:50.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4538143b80 0x7f4538144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453d297640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f453806c6c0 0x7f453806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4538143b80 0x7f4538144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:38280/0 (socket says 172.21.15.129:38280) 2024-04-25T00:45:50.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 -- 172.21.15.129:0/3471598572 learned_addr learned my addr 172.21.15.129:0/3471598572 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:50.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453ca96640 1 --1- 172.21.15.129:0/3471598572 >> v1:172.21.15.196:6789/0 conn(0x7f4538141560 0x7f4538072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:42274/0 (socket says 172.21.15.129:42274) 2024-04-25T00:45:50.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 -- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f453806c6c0 msgr2=0x7f453806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:50.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 --2- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f453806c6c0 0x7f453806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 -- 172.21.15.129:0/3471598572 >> v1:172.21.15.196:6789/0 conn(0x7f4538141560 legacy=0x7f4538072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f4537fff640 1 -- 172.21.15.129:0/3471598572 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4538071880 con 0x7f4538143b80 2024-04-25T00:45:50.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.791+0000 7f453d297640 1 --2- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f453806c6c0 0x7f453806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:50.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f4537fff640 1 --2- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538143b80 0x7f4538144000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4528009870 tx=0x7f452801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20f5180cf265cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f45377fe640 1 -- 172.21.15.129:0/3471598572 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45280b2070 con 0x7f4538143b80 2024-04-25T00:45:50.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f45377fe640 1 -- 172.21.15.129:0/3471598572 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45280a9070 con 0x7f4538143b80 2024-04-25T00:45:50.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f45377fe640 1 -- 172.21.15.129:0/3471598572 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45280ad880 con 0x7f4538143b80 2024-04-25T00:45:50.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 -- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538143b80 msgr2=0x7f4538144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 --2- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538143b80 0x7f4538144000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4528009870 tx=0x7f452801c560 comp rx=0 tx=0).stop 2024-04-25T00:45:50.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 -- 172.21.15.129:0/3471598572 shutdown_connections 2024-04-25T00:45:50.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 --2- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f453806c6c0 0x7f453806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 --2- 172.21.15.129:0/3471598572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538143b80 0x7f4538144000 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 -- 172.21.15.129:0/3471598572 >> 172.21.15.129:0/3471598572 conn(0x7f453806b210 msgr2=0x7f453806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 -- 172.21.15.129:0/3471598572 shutdown_connections 2024-04-25T00:45:50.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 -- 172.21.15.129:0/3471598572 wait complete. 2024-04-25T00:45:50.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 Processor -- start 2024-04-25T00:45:50.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.795+0000 7f453ecfa640 1 -- start start 2024-04-25T00:45:50.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 0x7f4538154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538141560 0x7f453815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4538143b80 0x7f45380d0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f453816b570 con 0x7f453806c6c0 2024-04-25T00:45:50.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f453816b3f0 con 0x7f4538141560 2024-04-25T00:45:50.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f453816b6f0 con 0x7f4538143b80 2024-04-25T00:45:50.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 0x7f4538154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 0x7f4538154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33944/0 (socket says 172.21.15.129:33944) 2024-04-25T00:45:50.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 -- 172.21.15.129:0/560624858 learned_addr learned my addr 172.21.15.129:0/560624858 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:50.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453d297640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4538143b80 0x7f45380d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f4537fff640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538141560 0x7f453815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 -- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4538143b80 msgr2=0x7f45380d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4538143b80 0x7f45380d0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 -- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538141560 msgr2=0x7f453815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538141560 0x7f453815fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 -- 172.21.15.129:0/560624858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45380d1700 con 0x7f453806c6c0 2024-04-25T00:45:50.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f4537fff640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538141560 0x7f453815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:50.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ca96640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 0x7f4538154730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c370 tx=0x7f452c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c0074e0 con 0x7f453806c6c0 2024-04-25T00:45:50.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c007680 con 0x7f453806c6c0 2024-04-25T00:45:50.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c004a30 con 0x7f453806c6c0 2024-04-25T00:45:50.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45380d19e0 con 0x7f453806c6c0 2024-04-25T00:45:50.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45380d1220 con 0x7f453806c6c0 2024-04-25T00:45:50.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4538062e50 con 0x7f453806c6c0 2024-04-25T00:45:50.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f452c01a020 con 0x7f453806c6c0 2024-04-25T00:45:50.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f45357fa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f45240678c0 0x7f4524069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:50.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f452c0854f0 con 0x7f453806c6c0 2024-04-25T00:45:50.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f4537fff640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f45240678c0 0x7f4524069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:50.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.799+0000 7f4537fff640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f45240678c0 0x7f4524069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45280af000 tx=0x7f45280a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:50.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.803+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f452c055f80 con 0x7f453806c6c0 2024-04-25T00:45:50.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.963+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4538075320 con 0x7f453806c6c0 2024-04-25T00:45:50.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.963+0000 7f45357fa640 1 -- 172.21.15.129:0/560624858 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f452c055880 con 0x7f453806c6c0 2024-04-25T00:45:50.966 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:45:50.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f45240678c0 msgr2=0x7f4524069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f45240678c0 0x7f4524069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45280af000 tx=0x7f45280a7000 comp rx=0 tx=0).stop 2024-04-25T00:45:50.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 msgr2=0x7f4538154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 0x7f4538154730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c370 tx=0x7f452c00c830 comp rx=0 tx=0).stop 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 shutdown_connections 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f45240678c0 0x7f4524069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4538143b80 0x7f45380d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4538141560 0x7f453815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 --2- 172.21.15.129:0/560624858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f453806c6c0 0x7f4538154730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 >> 172.21.15.129:0/560624858 conn(0x7f453806b210 msgr2=0x7f45380b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 shutdown_connections 2024-04-25T00:45:50.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:50.967+0000 7f453ecfa640 1 -- 172.21.15.129:0/560624858 wait complete. 2024-04-25T00:45:50.986 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:45:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb6f3e640 1 Processor -- start 2024-04-25T00:45:51.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb6f3e640 1 -- start start 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb6f3e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5cb015e470 0x7f5cb006e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb6f3e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb006cf00 con 0x7f5cb00cc690 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb6f3e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb006d1e0 con 0x7f5cb00b0e70 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb6f3e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb006d4c0 con 0x7f5cb015e470 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca7fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f5cb00cc690 0x7f5cb00b0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:35156/0 (socket says 172.21.15.129:35156) 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca7fff640 1 -- 172.21.15.129:0/2791733186 learned_addr learned my addr 172.21.15.129:0/2791733186 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:51.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/2791733186 >> v2:172.21.15.196:3300/0 conn(0x7f5cb015e470 0x7f5cb006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 180150423 0 0) 0x7f5cb006cf00 con 0x7f5cb00cc690 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c98003530 con 0x7f5cb00cc690 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1877005788 0 0) 0x7f5c98003530 con 0x7f5cb00cc690 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 >> v2:172.21.15.196:3300/0 conn(0x7f5cb015e470 msgr2=0x7f5cb006e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 --2- 172.21.15.129:0/2791733186 >> v2:172.21.15.196:3300/0 conn(0x7f5cb015e470 0x7f5cb006e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 >> v1:172.21.15.196:6789/0 conn(0x7f5cb00b0e70 legacy=0x7f5cb0070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb006d7a0 con 0x7f5cb00cc690 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f5ca0001680 con 0x7f5cb00cc690 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 >> v1:172.21.15.129:6789/0 conn(0x7f5cb00cc690 legacy=0x7f5cb00b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c980039c0 0x7f5c98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c98004810 0x7f5c98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980051d0 0x7f5c9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb006d4c0 con 0x7f5c980039c0 2024-04-25T00:45:51.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9800c170 con 0x7f5c98004810 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9800c370 con 0x7f5c980051d0 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.107+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c980039c0 0x7f5c98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5ca7fff640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c98004810 0x7f5c98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb54db640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980051d0 0x7f5c9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb4cda640 1 -- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980051d0 msgr2=0x7f5c9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980051d0 0x7f5c9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb4cda640 1 -- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c98004810 msgr2=0x7f5c98004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c98004810 0x7f5c98004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb4cda640 1 -- 172.21.15.129:0/2791733186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c98003530 con 0x7f5c980039c0 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb54db640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980051d0 0x7f5c9800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:51.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb54db640 1 -- 172.21.15.129:0/2791733186 reap_dead start 2024-04-25T00:45:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c980039c0 0x7f5c98003de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009ae0 tx=0x7f5cac01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf3f91176fbbbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:51.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0096b0 con 0x7f5c980039c0 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac0a8460 con 0x7f5c980039c0 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5ca77fe640 1 -- 172.21.15.129:0/2791733186 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0a8850 con 0x7f5c980039c0 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/2791733186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c980039c0 msgr2=0x7f5c98003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c980039c0 0x7f5c98003de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009ae0 tx=0x7f5cac01ca70 comp rx=0 tx=0).stop 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/2791733186 shutdown_connections 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/2791733186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c980039c0 0x7f5c98003de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/2791733186 >> 172.21.15.129:0/2791733186 conn(0x7f5cb00b14a0 msgr2=0x7f5cb00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/2791733186 shutdown_connections 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.111+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/2791733186 wait complete. 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 Processor -- start 2024-04-25T00:45:51.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- start start 2024-04-25T00:45:51.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980039c0 0x7f5cb014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 0x7f5cb0155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0155ec0 0x7f5cb0151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0162380 con 0x7f5cb0153850 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0162200 con 0x7f5cb0155ec0 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb0162500 con 0x7f5c980039c0 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 0x7f5cb0155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 0x7f5cb0155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33974/0 (socket says 172.21.15.129:33974) 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 -- 172.21.15.129:0/4201475946 learned_addr learned my addr 172.21.15.129:0/4201475946 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980039c0 0x7f5cb014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb54db640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0155ec0 0x7f5cb0151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 -- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980039c0 msgr2=0x7f5cb014e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980039c0 0x7f5cb014e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 -- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0155ec0 msgr2=0x7f5cb0151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0155ec0 0x7f5cb0151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 -- 172.21.15.129:0/4201475946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0151080 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca7fff640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 0x7f5cb0155980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000e7e0 tx=0x7f5ca000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000c8d0 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca0018070 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb0151360 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000ccd0 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb0151920 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb0062e50 con 0x7f5cb0153850 2024-04-25T00:45:51.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ca001d460 con 0x7f5cb0153850 2024-04-25T00:45:51.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.115+0000 7f5ca57fa640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f5c9c0678c0 0x7f5c9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:51.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.119+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f5ca0085bb0 con 0x7f5cb0153850 2024-04-25T00:45:51.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.119+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f5c9c0678c0 0x7f5c9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:51.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.119+0000 7f5cb4cda640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f5c9c0678c0 0x7f5c9c069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0af000 tx=0x7f5cac0046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:51.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.123+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5ca00566d0 con 0x7f5cb0153850 2024-04-25T00:45:51.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.279+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5cb00b0350 con 0x7f5cb0153850 2024-04-25T00:45:51.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.279+0000 7f5ca57fa640 1 -- 172.21.15.129:0/4201475946 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f5ca0055fd0 con 0x7f5cb0153850 2024-04-25T00:45:51.283 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:45:51.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f5c9c0678c0 msgr2=0x7f5c9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f5c9c0678c0 0x7f5c9c069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0af000 tx=0x7f5cac0046d0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 msgr2=0x7f5cb0155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 0x7f5cb0155980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000e7e0 tx=0x7f5ca000eca0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 shutdown_connections 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f5c9c0678c0 0x7f5c9c069d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0155ec0 0x7f5cb0151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cb0153850 0x7f5cb0155980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 --2- 172.21.15.129:0/4201475946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c980039c0 0x7f5cb014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 >> 172.21.15.129:0/4201475946 conn(0x7f5cb00b14a0 msgr2=0x7f5cb0074a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 shutdown_connections 2024-04-25T00:45:51.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:51.283+0000 7f5cb6f3e640 1 -- 172.21.15.129:0/4201475946 wait complete. 2024-04-25T00:45:51.301 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi129.front.sepia.ceph.com... 2024-04-25T00:45:51.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:51.301 DEBUG:teuthology.orchestra.run.smithi129:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:45:51.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[34950]: fuse finished with error 0 and tester_r 0 2024-04-25T00:45:51.382 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:45:57.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:45:57.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:57.485 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:45:57.500 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:45:57.500 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:57.552 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:45:57.553 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:57.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:45:57.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f450124d640 1 Processor -- start 2024-04-25T00:45:57.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f450124d640 1 -- start start 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f450124d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f44fc0b6bf0 0x7f44fc169130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f450124d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc15eac0 con 0x7f44fc1441b0 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f450124d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc15eda0 con 0x7f44fc144590 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f450124d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f44fc15f6d0 con 0x7f44fc0b6bf0 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fad9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f44fc1441b0 0x7f44fc1627b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34668/0 (socket says 172.21.15.129:34668) 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fad9d640 1 -- 172.21.15.129:0/4247418476 learned_addr learned my addr 172.21.15.129:0/4247418476 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fb59e640 1 --2- 172.21.15.129:0/4247418476 >> v2:172.21.15.196:3300/0 conn(0x7f44fc0b6bf0 0x7f44fc169130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763134813 0 0) 0x7f44fc15eac0 con 0x7f44fc1441b0 2024-04-25T00:45:57.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e8003530 con 0x7f44fc1441b0 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2866948991 0 0) 0x7f44e8003530 con 0x7f44fc1441b0 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 >> v2:172.21.15.196:3300/0 conn(0x7f44fc0b6bf0 msgr2=0x7f44fc169130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 --2- 172.21.15.129:0/4247418476 >> v2:172.21.15.196:3300/0 conn(0x7f44fc0b6bf0 0x7f44fc169130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 >> v1:172.21.15.196:6789/0 conn(0x7f44fc144590 legacy=0x7f44fc0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc15f080 con 0x7f44fc1441b0 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f44ec001680 con 0x7f44fc1441b0 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 >> v1:172.21.15.129:6789/0 conn(0x7f44fc1441b0 legacy=0x7f44fc1627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e80039c0 0x7f44e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e80051d0 0x7f44e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc15f6d0 con 0x7f44e8004810 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44e800c0a0 con 0x7f44e80039c0 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44e800c2a0 con 0x7f44e80051d0 2024-04-25T00:45:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44f259c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fad9d640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e80039c0 0x7f44e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fb59e640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e80051d0 0x7f44e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44f259c640 1 -- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e80051d0 msgr2=0x7f44e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44f259c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e80051d0 0x7f44e800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44f259c640 1 -- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e80039c0 msgr2=0x7f44e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44f259c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44e80039c0 0x7f44e8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44f259c640 1 -- 172.21.15.129:0/4247418476 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e8003530 con 0x7f44e8004810 2024-04-25T00:45:57.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.771+0000 7f44fb59e640 1 -- 172.21.15.129:0/4247418476 reap_dead start 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44f259c640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f44e4001780 tx=0x7f44e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a81355927a749bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40013a0 con 0x7f44e8004810 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e40052f0 con 0x7f44e8004810 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fa59c640 1 -- 172.21.15.129:0/4247418476 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003840 con 0x7f44e8004810 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- 172.21.15.129:0/4247418476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44e8004810 msgr2=0x7f44e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f44e4001780 tx=0x7f44e401c400 comp rx=0 tx=0).stop 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- 172.21.15.129:0/4247418476 shutdown_connections 2024-04-25T00:45:57.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 --2- 172.21.15.129:0/4247418476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44e8004810 0x7f44e8004c90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- 172.21.15.129:0/4247418476 >> 172.21.15.129:0/4247418476 conn(0x7f44fc0b52d0 msgr2=0x7f44fc0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:57.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- 172.21.15.129:0/4247418476 shutdown_connections 2024-04-25T00:45:57.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- 172.21.15.129:0/4247418476 wait complete. 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 Processor -- start 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- start start 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e8004810 0x7f44fc15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc140080 0x7f44fc15be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 0x7f44fc157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc143a90 con 0x7f44fc15c370 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc15e760 con 0x7f44fc140080 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f450124d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc143c10 con 0x7f44e8004810 2024-04-25T00:45:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fb59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 0x7f44fc157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fb59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 0x7f44fc157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42490/0 (socket says 172.21.15.129:42490) 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fb59e640 1 -- 172.21.15.129:0/2217697172 learned_addr learned my addr 172.21.15.129:0/2217697172 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44f259c640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc140080 0x7f44fc15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.775+0000 7f44fad9d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e8004810 0x7f44fc15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fb59e640 1 -- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e8004810 msgr2=0x7f44fc15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fb59e640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e8004810 0x7f44fc15d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fb59e640 1 -- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc140080 msgr2=0x7f44fc15be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fb59e640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc140080 0x7f44fc15be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fb59e640 1 -- 172.21.15.129:0/2217697172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc15c8d0 con 0x7f44fc15c370 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fb59e640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 0x7f44fc157cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0b34e0 tx=0x7f44f400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4016420 con 0x7f44fc15c370 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f401b070 con 0x7f44fc15c370 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc15cbb0 con 0x7f44fc15c370 2024-04-25T00:45:57.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4016840 con 0x7f44fc15c370 2024-04-25T00:45:57.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc15d170 con 0x7f44fc15c370 2024-04-25T00:45:57.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc062e50 con 0x7f44fc15c370 2024-04-25T00:45:57.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f44f40169e0 con 0x7f44fc15c370 2024-04-25T00:45:57.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44f3fff640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f44dc0678f0 0x7f44dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:57.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f44f4086d50 con 0x7f44fc15c370 2024-04-25T00:45:57.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fad9d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f44dc0678f0 0x7f44dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:57.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.779+0000 7f44fad9d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f44dc0678f0 0x7f44dc069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44ec001000 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:57.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.783+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f44f408a030 con 0x7f44fc15c370 2024-04-25T00:45:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.947+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44fc15e280 con 0x7f44fc15c370 2024-04-25T00:45:57.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.947+0000 7f44f3fff640 1 -- 172.21.15.129:0/2217697172 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f44f4025020 con 0x7f44fc15c370 2024-04-25T00:45:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:45:57.951 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f44dc0678f0 msgr2=0x7f44dc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f44dc0678f0 0x7f44dc069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44ec001000 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 msgr2=0x7f44fc157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 0x7f44fc157cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0b34e0 tx=0x7f44f400f520 comp rx=0 tx=0).stop 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 shutdown_connections 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f44dc0678f0 0x7f44dc069db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44fc15c370 0x7f44fc157cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44fc140080 0x7f44fc15be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 --2- 172.21.15.129:0/2217697172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44e8004810 0x7f44fc15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 >> 172.21.15.129:0/2217697172 conn(0x7f44fc0b52d0 msgr2=0x7f44fc1408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 shutdown_connections 2024-04-25T00:45:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:57.951+0000 7f450124d640 1 -- 172.21.15.129:0/2217697172 wait complete. 2024-04-25T00:45:57.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:45:57.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:45:58.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f961e5640 1 Processor -- start 2024-04-25T00:45:58.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f961e5640 1 -- start start 2024-04-25T00:45:58.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f961e5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 0x7f2f90143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f961e5640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f900d2710 con 0x7f2f90141010 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f961e5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f900d29c0 con 0x7f2f90143e80 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f961e5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2f900d2ca0 con 0x7f2f90144570 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8f7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f2f90141010 0x7f2f900d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34676/0 (socket says 172.21.15.129:34676) 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8f7fe640 1 -- 172.21.15.129:0/3839392217 learned_addr learned my addr 172.21.15.129:0/3839392217 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 0x7f2f90143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643829588 0 0) 0x7f2f900d2710 con 0x7f2f90141010 2024-04-25T00:45:58.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f78003530 con 0x7f2f90141010 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 952080497 0 0) 0x7f2f78003530 con 0x7f2f90141010 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 msgr2=0x7f2f90143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 0x7f2f90143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 >> v1:172.21.15.196:6789/0 conn(0x7f2f90143e80 legacy=0x7f2f901418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f900d2f80 con 0x7f2f90141010 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f2f88001a00 con 0x7f2f90141010 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 >> v1:172.21.15.129:6789/0 conn(0x7f2f90141010 legacy=0x7f2f900d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f780039f0 0x7f2f78003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f78004810 0x7f2f78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 0x7f2f78005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f900d2ca0 con 0x7f2f78004810 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7800bb90 con 0x7f2f780039f0 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7800bd90 con 0x7f2f90144570 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f78004810 0x7f2f78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8f7fe640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f780039f0 0x7f2f78003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 -- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 msgr2=0x7f2f78005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90144570 0x7f2f78005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 -- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f780039f0 msgr2=0x7f2f78003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f780039f0 0x7f2f78003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.087+0000 7f2f8effd640 1 -- 172.21.15.129:0/3839392217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f78003530 con 0x7f2f78004810 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f8effd640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f78004810 0x7f2f78004c90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f900b3840 tx=0x7f2f8401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12116a89a28d7c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84009690 con 0x7f2f78004810 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f840a6410 con 0x7f2f78004810 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f8e7fc640 1 -- 172.21.15.129:0/3839392217 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f840a67e0 con 0x7f2f78004810 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 -- 172.21.15.129:0/3839392217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f78004810 msgr2=0x7f2f78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 --2- 172.21.15.129:0/3839392217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f78004810 0x7f2f78004c90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f900b3840 tx=0x7f2f8401ca30 comp rx=0 tx=0).stop 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f8ffff640 1 -- 172.21.15.129:0/3839392217 reap_dead start 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 -- 172.21.15.129:0/3839392217 shutdown_connections 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 -- 172.21.15.129:0/3839392217 >> 172.21.15.129:0/3839392217 conn(0x7f2f90072be0 msgr2=0x7f2f900d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 -- 172.21.15.129:0/3839392217 shutdown_connections 2024-04-25T00:45:58.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 -- 172.21.15.129:0/3839392217 wait complete. 2024-04-25T00:45:58.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 Processor -- start 2024-04-25T00:45:58.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.091+0000 7f2f961e5640 1 -- start start 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f78004810 0x7f2f900b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90071680 0x7f2f90071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 0x7f2f900b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90142950 con 0x7f2f900b2170 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f901427d0 con 0x7f2f78004810 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2f90142ad0 con 0x7f2f90071680 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 0x7f2f900b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 0x7f2f900b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42500/0 (socket says 172.21.15.129:42500) 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 -- 172.21.15.129:0/1868634351 learned_addr learned my addr 172.21.15.129:0/1868634351 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8f7fe640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f78004810 0x7f2f900b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8effd640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90071680 0x7f2f90071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 -- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90071680 msgr2=0x7f2f90071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90071680 0x7f2f90071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 -- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f78004810 msgr2=0x7f2f900b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f78004810 0x7f2f900b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 -- 172.21.15.129:0/1868634351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f900b0d10 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8f7fe640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f78004810 0x7f2f900b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8ffff640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 0x7f2f900b25f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f900ae880 tx=0x7f2f8000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000cea0 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80007520 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901fbee0 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80004260 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f900ac880 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f90062e50 con 0x7f2f900b2170 2024-04-25T00:45:58.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2f800057d0 con 0x7f2f900b2170 2024-04-25T00:45:58.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f77fff640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2f7c0678f0 0x7f2f7c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f2f80084e20 con 0x7f2f900b2170 2024-04-25T00:45:58.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8f7fe640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2f7c0678f0 0x7f2f7c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.095+0000 7f2f8f7fe640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2f7c0678f0 0x7f2f7c069db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f88007c50 tx=0x7f2f88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.099+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f900b0d10 con 0x7f2f900b2170 2024-04-25T00:45:58.160 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:45:58.163 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:45:58.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.259+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f90164f00 con 0x7f2f900b2170 2024-04-25T00:45:58.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.259+0000 7f2f77fff640 1 -- 172.21.15.129:0/1868634351 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f2f80055940 con 0x7f2f900b2170 2024-04-25T00:45:58.262 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:45:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.259+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2f7c0678f0 msgr2=0x7f2f7c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.259+0000 7f2f961e5640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2f7c0678f0 0x7f2f7c069db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f88007c50 tx=0x7f2f88009040 comp rx=0 tx=0).stop 2024-04-25T00:45:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.259+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 msgr2=0x7f2f900b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.259+0000 7f2f961e5640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 0x7f2f900b25f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f900ae880 tx=0x7f2f8000c820 comp rx=0 tx=0).stop 2024-04-25T00:45:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 shutdown_connections 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2f7c0678f0 0x7f2f7c069db0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2f90071680 0x7f2f90071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f900b2170 0x7f2f900b25f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 --2- 172.21.15.129:0/1868634351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f78004810 0x7f2f900b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 >> 172.21.15.129:0/1868634351 conn(0x7f2f90072be0 msgr2=0x7f2f90160e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 shutdown_connections 2024-04-25T00:45:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.263+0000 7f2f961e5640 1 -- 172.21.15.129:0/1868634351 wait complete. 2024-04-25T00:45:58.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:45:58.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.399+0000 7f1a3ff41640 1 Processor -- start 2024-04-25T00:45:58.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.399+0000 7f1a3ff41640 1 -- start start 2024-04-25T00:45:58.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1a3815bd90 0x7f1a380cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a380cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a380ccb80 con 0x7f1a3815c170 2024-04-25T00:45:58.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f1a380cc930 con 0x7f1a3815bd90 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1a380cd4b0 con 0x7f1a3806fa80 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3d4dc640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f1a3815c170 0x7f1a3806f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34684/0 (socket says 172.21.15.129:34684) 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3dcdd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1a3815bd90 0x7f1a380cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3d4dc640 1 -- 172.21.15.129:0/2385900954 learned_addr learned my addr 172.21.15.129:0/2385900954 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3e4de640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a380cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3dcdd640 1 -- 172.21.15.129:0/2385900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 msgr2=0x7f1a380cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3dcdd640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a380cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3dcdd640 1 -- 172.21.15.129:0/2385900954 >> v1:172.21.15.129:6789/0 conn(0x7f1a3815c170 legacy=0x7f1a3806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3dcdd640 1 -- 172.21.15.129:0/2385900954 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a380cce60 con 0x7f1a3815bd90 2024-04-25T00:45:58.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3e4de640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a380cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:58.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3dcdd640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815bd90 0x7f1a380cf060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a280097c0 tx=0x7f1a2801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3e2dfa0782f415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ccdb640 1 -- 172.21.15.129:0/2385900954 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280b0070 con 0x7f1a3815bd90 2024-04-25T00:45:58.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ccdb640 1 -- 172.21.15.129:0/2385900954 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a280ab600 con 0x7f1a3815bd90 2024-04-25T00:45:58.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ccdb640 1 -- 172.21.15.129:0/2385900954 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280ab9d0 con 0x7f1a3815bd90 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 -- 172.21.15.129:0/2385900954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815bd90 msgr2=0x7f1a380cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.403+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815bd90 0x7f1a380cf060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a280097c0 tx=0x7f1a2801c410 comp rx=0 tx=0).stop 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/2385900954 shutdown_connections 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a380cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/2385900954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815bd90 0x7f1a380cf060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/2385900954 >> 172.21.15.129:0/2385900954 conn(0x7f1a38067730 msgr2=0x7f1a380b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/2385900954 shutdown_connections 2024-04-25T00:45:58.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/2385900954 wait complete. 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 Processor -- start 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- start start 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a3815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3815bd90 0x7f1a38148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 0x7f1a3813d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3806d820 con 0x7f1a3815c170 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3806d6a0 con 0x7f1a3815bd90 2024-04-25T00:45:58.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3806d9a0 con 0x7f1a3806fa80 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 0x7f1a3813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 0x7f1a3813d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42510/0 (socket says 172.21.15.129:42510) 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 -- 172.21.15.129:0/4196166856 learned_addr learned my addr 172.21.15.129:0/4196166856 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3dcdd640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a3815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3d4dc640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3815bd90 0x7f1a38148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 -- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 msgr2=0x7f1a3815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a3815a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 -- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3815bd90 msgr2=0x7f1a38148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3815bd90 0x7f1a38148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 -- 172.21.15.129:0/4196166856 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a38148d30 con 0x7f1a3815c170 2024-04-25T00:45:58.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3e4de640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 0x7f1a3813d500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000cd50 tx=0x7f1a3000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000f680 con 0x7f1a3815c170 2024-04-25T00:45:58.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3000f840 con 0x7f1a3815c170 2024-04-25T00:45:58.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a38149010 con 0x7f1a3815c170 2024-04-25T00:45:58.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000fc30 con 0x7f1a3815c170 2024-04-25T00:45:58.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a381495d0 con 0x7f1a3815c170 2024-04-25T00:45:58.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.407+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a38062e50 con 0x7f1a3815c170 2024-04-25T00:45:58.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.411+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1a300213f0 con 0x7f1a3815c170 2024-04-25T00:45:58.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.411+0000 7f1a2e7fc640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1a1c0678c0 0x7f1a1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.411+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f1a30086c50 con 0x7f1a3815c170 2024-04-25T00:45:58.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.411+0000 7f1a3dcdd640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1a1c0678c0 0x7f1a1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.411+0000 7f1a3dcdd640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1a1c0678c0 0x7f1a1c069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a280ad000 tx=0x7f1a28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.415+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a30014030 con 0x7f1a3815c170 2024-04-25T00:45:58.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.559+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a38070840 con 0x7f1a3815c170 2024-04-25T00:45:58.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.559+0000 7f1a2e7fc640 1 -- 172.21.15.129:0/4196166856 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1a300575b0 con 0x7f1a3815c170 2024-04-25T00:45:58.563 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:45:58.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1a1c0678c0 msgr2=0x7f1a1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1a1c0678c0 0x7f1a1c069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a280ad000 tx=0x7f1a28008040 comp rx=0 tx=0).stop 2024-04-25T00:45:58.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 msgr2=0x7f1a3813d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 0x7f1a3813d500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000cd50 tx=0x7f1a3000a650 comp rx=0 tx=0).stop 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 shutdown_connections 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1a1c0678c0 0x7f1a1c069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3815c170 0x7f1a3813d500 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a3815bd90 0x7f1a38148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 --2- 172.21.15.129:0/4196166856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a3806fa80 0x7f1a3815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 >> 172.21.15.129:0/4196166856 conn(0x7f1a38067730 msgr2=0x7f1a3815b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 shutdown_connections 2024-04-25T00:45:58.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.563+0000 7f1a3ff41640 1 -- 172.21.15.129:0/4196166856 wait complete. 2024-04-25T00:45:58.580 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:45:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 Processor -- start 2024-04-25T00:45:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 -- start start 2024-04-25T00:45:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f09b40cffc0 0x7f09b415f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b40cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b40b2f10 con 0x7f09b40b4660 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f09b415f860 con 0x7f09b40cffc0 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b9a43640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09b40b3180 con 0x7f09b40af190 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f09b40cffc0 0x7f09b415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f09b40cffc0 0x7f09b415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42526/0 (socket says 172.21.15.129:42526) 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 -- 172.21.15.129:0/93776632 learned_addr learned my addr 172.21.15.129:0/93776632 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b27fc640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f09b40b4660 0x7f09b40aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34686/0 (socket says 172.21.15.129:34686) 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b37fe640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b40cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 -- 172.21.15.129:0/93776632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 msgr2=0x7f09b40cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b40cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 -- 172.21.15.129:0/93776632 >> v1:172.21.15.129:6789/0 conn(0x7f09b40b4660 legacy=0x7f09b40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b2ffd640 1 -- 172.21.15.129:0/93776632 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b40b3460 con 0x7f09b40cffc0 2024-04-25T00:45:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b1ffb640 1 -- 172.21.15.129:0/93776632 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519214204 0 0) 0x7f09b40b2f10 con 0x7f09b40b4660 2024-04-25T00:45:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.695+0000 7f09b37fe640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b40cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b2ffd640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40cffc0 0x7f09b415f320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009870 tx=0x7f09a801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b86a1f6d9d79b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b1ffb640 1 -- 172.21.15.129:0/93776632 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80ab640 con 0x7f09b40cffc0 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b1ffb640 1 -- 172.21.15.129:0/93776632 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a80b0070 con 0x7f09b40cffc0 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b1ffb640 1 -- 172.21.15.129:0/93776632 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80aba00 con 0x7f09b40cffc0 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- 172.21.15.129:0/93776632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40cffc0 msgr2=0x7f09b415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40cffc0 0x7f09b415f320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009870 tx=0x7f09a801c530 comp rx=0 tx=0).stop 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- 172.21.15.129:0/93776632 shutdown_connections 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b40cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 --2- 172.21.15.129:0/93776632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40cffc0 0x7f09b415f320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- 172.21.15.129:0/93776632 >> 172.21.15.129:0/93776632 conn(0x7f09b4071ac0 msgr2=0x7f09b4071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- 172.21.15.129:0/93776632 shutdown_connections 2024-04-25T00:45:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- 172.21.15.129:0/93776632 wait complete. 2024-04-25T00:45:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 Processor -- start 2024-04-25T00:45:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- start start 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 0x7f09b40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b4150fb0 0x7f09b413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40d0520 con 0x7f09b40b4660 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09b40d03a0 con 0x7f09b4150fb0 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09b40cdb60 con 0x7f09b40af190 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 0x7f09b40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 0x7f09b40cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42532/0 (socket says 172.21.15.129:42532) 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 -- 172.21.15.129:0/1413582308 learned_addr learned my addr 172.21.15.129:0/1413582308 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 -- 172.21.15.129:0/1413582308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 msgr2=0x7f09b414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 -- 172.21.15.129:0/1413582308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b4150fb0 msgr2=0x7f09b413fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b4150fb0 0x7f09b413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 -- 172.21.15.129:0/1413582308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b414fae0 con 0x7f09b40b4660 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b27fc640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 0x7f09b40cedf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c410 tx=0x7f09a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40074e0 con 0x7f09b40b4660 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a40047f0 con 0x7f09b40b4660 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007680 con 0x7f09b40b4660 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.699+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b414fdc0 con 0x7f09b40b4660 2024-04-25T00:45:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b4150380 con 0x7f09b40b4660 2024-04-25T00:45:58.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b4062e50 con 0x7f09b40b4660 2024-04-25T00:45:58.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f09a4011070 con 0x7f09b40b4660 2024-04-25T00:45:58.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f099bfff640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09a00678f0 0x7f09a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f09a40858a0 con 0x7f09b40b4660 2024-04-25T00:45:58.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f09b2ffd640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09a00678f0 0x7f09a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.703+0000 7f09b2ffd640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09a00678f0 0x7f09a0069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09a80af000 tx=0x7f09a8004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.707+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f09a40563c0 con 0x7f09b40b4660 2024-04-25T00:45:58.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.855+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f09b415ea30 con 0x7f09b40b4660 2024-04-25T00:45:58.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.855+0000 7f099bfff640 1 -- 172.21.15.129:0/1413582308 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f09a4055cc0 con 0x7f09b40b4660 2024-04-25T00:45:58.859 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:45:58.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09a00678f0 msgr2=0x7f09a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09a00678f0 0x7f09a0069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09a80af000 tx=0x7f09a8004740 comp rx=0 tx=0).stop 2024-04-25T00:45:58.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 msgr2=0x7f09b40cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 0x7f09b40cedf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c410 tx=0x7f09a400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 shutdown_connections 2024-04-25T00:45:58.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09a00678f0 0x7f09a0069db0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09b4150fb0 0x7f09b413fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40b4660 0x7f09b40cedf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 --2- 172.21.15.129:0/1413582308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09b40af190 0x7f09b414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 >> 172.21.15.129:0/1413582308 conn(0x7f09b4071ac0 msgr2=0x7f09b40aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 shutdown_connections 2024-04-25T00:45:58.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.859+0000 7f09b9a43640 1 -- 172.21.15.129:0/1413582308 wait complete. 2024-04-25T00:45:58.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:45:58.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b87099640 1 Processor -- start 2024-04-25T00:45:58.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b87099640 1 -- start start 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b87099640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f4b80070de0 0x7f4b80166cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b87099640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b800b2a90 con 0x7f4b8006e820 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b87099640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8015ce70 con 0x7f4b800b26b0 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b87099640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8015d150 con 0x7f4b80070de0 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b85636640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f4b80070de0 0x7f4b80166cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b7ffff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f4b8006e820 0x7f4b800706d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34698/0 (socket says 172.21.15.129:34698) 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1796606524 learned_addr learned my addr 172.21.15.129:0/1796606524 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b85636640 1 -- 172.21.15.129:0/1796606524 >> v1:172.21.15.196:6789/0 conn(0x7f4b800b26b0 legacy=0x7f4b8015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b85636640 1 -- 172.21.15.129:0/1796606524 >> v1:172.21.15.129:6789/0 conn(0x7f4b8006e820 legacy=0x7f4b800706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b85636640 1 -- 172.21.15.129:0/1796606524 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8015d430 con 0x7f4b80070de0 2024-04-25T00:45:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.995+0000 7f4b7f7fe640 1 -- 172.21.15.129:0/1796606524 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802465409 0 0) 0x7f4b800b2a90 con 0x7f4b8006e820 2024-04-25T00:45:58.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 --2- 172.21.15.129:0/1796606524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b80070de0 0x7f4b80166cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d90 tx=0x7f4b7801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bca7ef3477fbb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:58.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7f7fe640 1 -- 172.21.15.129:0/1796606524 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780a66f0 con 0x7f4b80070de0 2024-04-25T00:45:58.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7f7fe640 1 -- 172.21.15.129:0/1796606524 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b780a6890 con 0x7f4b80070de0 2024-04-25T00:45:58.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7f7fe640 1 -- 172.21.15.129:0/1796606524 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780a6c60 con 0x7f4b80070de0 2024-04-25T00:45:59.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- 172.21.15.129:0/1796606524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b80070de0 msgr2=0x7f4b80166cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 --2- 172.21.15.129:0/1796606524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b80070de0 0x7f4b80166cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d90 tx=0x7f4b7801c930 comp rx=0 tx=0).stop 2024-04-25T00:45:59.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- 172.21.15.129:0/1796606524 shutdown_connections 2024-04-25T00:45:59.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 --2- 172.21.15.129:0/1796606524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b80070de0 0x7f4b80166cb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- 172.21.15.129:0/1796606524 >> 172.21.15.129:0/1796606524 conn(0x7f4b80067070 msgr2=0x7f4b80067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- 172.21.15.129:0/1796606524 shutdown_connections 2024-04-25T00:45:59.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- 172.21.15.129:0/1796606524 wait complete. 2024-04-25T00:45:59.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 Processor -- start 2024-04-25T00:45:59.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- start start 2024-04-25T00:45:59.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8006e820 0x7f4b801537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b80070de0 0x7f4b80152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 0x7f4b80072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80169da0 con 0x7f4b800b26b0 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b80169c20 con 0x7f4b80070de0 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b80169f20 con 0x7f4b8006e820 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 0x7f4b80072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 0x7f4b80072b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42554/0 (socket says 172.21.15.129:42554) 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 -- 172.21.15.129:0/4012928551 learned_addr learned my addr 172.21.15.129:0/4012928551 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b84e35640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8006e820 0x7f4b801537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7ffff640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b80070de0 0x7f4b80152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 -- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8006e820 msgr2=0x7f4b801537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8006e820 0x7f4b801537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 -- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b80070de0 msgr2=0x7f4b80152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b80070de0 0x7f4b80152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 -- 172.21.15.129:0/4012928551 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b800d0e50 con 0x7f4b800b26b0 2024-04-25T00:45:59.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b85636640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 0x7f4b80072b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b780b0040 tx=0x7f4b780196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780aebe0 con 0x7f4b800b26b0 2024-04-25T00:45:59.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b780c1070 con 0x7f4b800b26b0 2024-04-25T00:45:59.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b800d0c20 con 0x7f4b800b26b0 2024-04-25T00:45:59.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:58.999+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780a66b0 con 0x7f4b800b26b0 2024-04-25T00:45:59.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b80071e80 con 0x7f4b800b26b0 2024-04-25T00:45:59.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b80062e50 con 0x7f4b800b26b0 2024-04-25T00:45:59.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b780c6460 con 0x7f4b800b26b0 2024-04-25T00:45:59.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b7d7fa640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4b6c0678c0 0x7f4b6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f4b781212f0 con 0x7f4b800b26b0 2024-04-25T00:45:59.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b84e35640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4b6c0678c0 0x7f4b6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.003+0000 7f4b84e35640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4b6c0678c0 0x7f4b6c069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b80156750 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.007+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4b78016520 con 0x7f4b800b26b0 2024-04-25T00:45:59.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.167+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b80075320 con 0x7f4b800b26b0 2024-04-25T00:45:59.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.167+0000 7f4b7d7fa640 1 -- 172.21.15.129:0/4012928551 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f4b780f1ed0 con 0x7f4b800b26b0 2024-04-25T00:45:59.169 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:45:59.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4b6c0678c0 msgr2=0x7f4b6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4b6c0678c0 0x7f4b6c069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b80156750 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 msgr2=0x7f4b80072b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 0x7f4b80072b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b780b0040 tx=0x7f4b780196d0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 shutdown_connections 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4b6c0678c0 0x7f4b6c069d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b800b26b0 0x7f4b80072b30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b80070de0 0x7f4b80152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 --2- 172.21.15.129:0/4012928551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8006e820 0x7f4b801537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 >> 172.21.15.129:0/4012928551 conn(0x7f4b80067070 msgr2=0x7f4b8015dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 shutdown_connections 2024-04-25T00:45:59.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.171+0000 7f4b87099640 1 -- 172.21.15.129:0/4012928551 wait complete. 2024-04-25T00:45:59.187 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:45:59.187 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:59.193 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:45:59.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ad14ee640 1 Processor -- start 2024-04-25T00:45:59.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ad14ee640 1 -- start start 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ad14ee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1acc0cd190 0x7f1acc0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ad14ee640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1acc072aa0 con 0x7f1acc0ccdb0 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ad14ee640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1acc0708e0 con 0x7f1acc15c7f0 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ad14ee640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1acc070690 con 0x7f1acc0cd190 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1acaffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f1acc0ccdb0 0x7f1acc158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34708/0 (socket says 172.21.15.129:34708) 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1acaffd640 1 -- 172.21.15.129:0/815580149 learned_addr learned my addr 172.21.15.129:0/815580149 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1acb7fe640 1 --2- 172.21.15.129:0/815580149 >> v2:172.21.15.196:3300/0 conn(0x7f1acc0cd190 0x7f1acc0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217352391 0 0) 0x7f1acc072aa0 con 0x7f1acc0ccdb0 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ab4003530 con 0x7f1acc0ccdb0 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1085216375 0 0) 0x7f1ab4003530 con 0x7f1acc0ccdb0 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 >> v2:172.21.15.196:3300/0 conn(0x7f1acc0cd190 msgr2=0x7f1acc0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:59.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 --2- 172.21.15.129:0/815580149 >> v2:172.21.15.196:3300/0 conn(0x7f1acc0cd190 0x7f1acc0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 >> v1:172.21.15.196:6789/0 conn(0x7f1acc15c7f0 legacy=0x7f1acc06de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc070bc0 con 0x7f1acc0ccdb0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f1abc001680 con 0x7f1acc0ccdb0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 >> v1:172.21.15.129:6789/0 conn(0x7f1acc0ccdb0 legacy=0x7f1acc158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ab40039c0 0x7f1ab4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ab4004810 0x7f1ab4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab40051d0 0x7f1ab400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc070690 con 0x7f1ab4004810 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab400c170 con 0x7f1ab40051d0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab400c370 con 0x7f1ab40039c0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ab4004810 0x7f1ab4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1acb7fe640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab40051d0 0x7f1ab400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1acaffd640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ab40039c0 0x7f1ab4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1aca7fc640 1 -- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ab40039c0 msgr2=0x7f1ab4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:59.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ab40039c0 0x7f1ab4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1aca7fc640 1 -- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab40051d0 msgr2=0x7f1ab400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:59.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab40051d0 0x7f1ab400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1acb7fe640 1 -- 172.21.15.129:0/815580149 reap_dead start 2024-04-25T00:45:59.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.351+0000 7f1aca7fc640 1 -- 172.21.15.129:0/815580149 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab4003530 con 0x7f1ab4004810 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ab4004810 0x7f1ab4004c90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0001360 tx=0x7f1ac001e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ada6bb1685a2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00013a0 con 0x7f1ab4004810 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac00052f0 con 0x7f1ab4004810 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ac9ffb640 1 -- 172.21.15.129:0/815580149 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0003840 con 0x7f1ab4004810 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 -- 172.21.15.129:0/815580149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ab4004810 msgr2=0x7f1ab4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ab4004810 0x7f1ab4004c90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0001360 tx=0x7f1ac001e610 comp rx=0 tx=0).stop 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 -- 172.21.15.129:0/815580149 shutdown_connections 2024-04-25T00:45:59.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/815580149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ab4004810 0x7f1ab4004c90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 -- 172.21.15.129:0/815580149 >> 172.21.15.129:0/815580149 conn(0x7f1acc0b14a0 msgr2=0x7f1acc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 -- 172.21.15.129:0/815580149 shutdown_connections 2024-04-25T00:45:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 -- 172.21.15.129:0/815580149 wait complete. 2024-04-25T00:45:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 Processor -- start 2024-04-25T00:45:59.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 -- start start 2024-04-25T00:45:59.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab4004810 0x7f1acc146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 0x7f1acc14a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.355+0000 7f1ad14ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1acc1443e0 0x7f1acc144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ad14ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0713a0 con 0x7f1acc149e60 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ad14ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc159060 con 0x7f1ab4004810 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ad14ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1acc071520 con 0x7f1acc1443e0 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 0x7f1acc14a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 0x7f1acc14a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42562/0 (socket says 172.21.15.129:42562) 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 -- 172.21.15.129:0/3232849562 learned_addr learned my addr 172.21.15.129:0/3232849562 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1acb7fe640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1acc1443e0 0x7f1acc144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1acaffd640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab4004810 0x7f1acc146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 -- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1acc1443e0 msgr2=0x7f1acc144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1acc1443e0 0x7f1acc144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 -- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab4004810 msgr2=0x7f1acc146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab4004810 0x7f1acc146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 -- 172.21.15.129:0/3232849562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc145150 con 0x7f1acc149e60 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1acb7fe640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1acc1443e0 0x7f1acc144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1aca7fc640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 0x7f1acc14a2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0099e10 tx=0x7f1ab4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007a40 con 0x7f1acc149e60 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac00bc070 con 0x7f1acc149e60 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007250 con 0x7f1acc149e60 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc145430 con 0x7f1acc149e60 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1459f0 con 0x7f1acc149e60 2024-04-25T00:45:59.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc062e50 con 0x7f1acc149e60 2024-04-25T00:45:59.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ac00052f0 con 0x7f1acc149e60 2024-04-25T00:45:59.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ab37fe640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ab80678c0 0x7f1ab8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f1ac0121e60 con 0x7f1acc149e60 2024-04-25T00:45:59.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1acaffd640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ab80678c0 0x7f1ab8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.359+0000 7f1acaffd640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ab80678c0 0x7f1ab8069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1acc067040 tx=0x7f1abc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.363+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ac0018320 con 0x7f1acc149e60 2024-04-25T00:45:59.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.511+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1acc075320 con 0x7f1acc149e60 2024-04-25T00:45:59.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ab37fe640 1 -- 172.21.15.129:0/3232849562 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f1ac00f28f0 con 0x7f1acc149e60 2024-04-25T00:45:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:45:59.516 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ab80678c0 msgr2=0x7f1ab8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ab80678c0 0x7f1ab8069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1acc067040 tx=0x7f1abc009040 comp rx=0 tx=0).stop 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 msgr2=0x7f1acc14a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 0x7f1acc14a2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0099e10 tx=0x7f1ab4005840 comp rx=0 tx=0).stop 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 shutdown_connections 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1ab80678c0 0x7f1ab8069d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1acc1443e0 0x7f1acc144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1acc149e60 0x7f1acc14a2e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 --2- 172.21.15.129:0/3232849562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab4004810 0x7f1acc146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 >> 172.21.15.129:0/3232849562 conn(0x7f1acc0b14a0 msgr2=0x7f1acc0cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 shutdown_connections 2024-04-25T00:45:59.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.515+0000 7f1ad14ee640 1 -- 172.21.15.129:0/3232849562 wait complete. 2024-04-25T00:45:59.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:45:59.534 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:45:59.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 Processor -- start 2024-04-25T00:45:59.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 -- start start 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8884145960 0x7f8884160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f888415c590 0x7f88840cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f888406dbf0 con 0x7f88840cccd0 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f88840cd140 con 0x7f888415c590 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8889425640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8884160ad0 con 0x7f8884145960 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f888415c590 0x7f88840cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f888415c590 0x7f88840cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42572/0 (socket says 172.21.15.129:42572) 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 -- 172.21.15.129:0/1655827739 learned_addr learned my addr 172.21.15.129:0/1655827739 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f8882ffd640 1 --2- 172.21.15.129:0/1655827739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8884145960 0x7f8884160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 -- 172.21.15.129:0/1655827739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8884145960 msgr2=0x7f8884160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:59.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 --2- 172.21.15.129:0/1655827739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8884145960 0x7f8884160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 -- 172.21.15.129:0/1655827739 >> v1:172.21.15.196:6789/0 conn(0x7f88840cccd0 legacy=0x7f88840cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:59.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 -- 172.21.15.129:0/1655827739 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888406f090 con 0x7f888415c590 2024-04-25T00:45:59.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.651+0000 7f88827fc640 1 --2- 172.21.15.129:0/1655827739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f888415c590 0x7f88840cc790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f886c009870 tx=0x7f886c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecadddf56320d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8881ffb640 1 -- 172.21.15.129:0/1655827739 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c0b0070 con 0x7f888415c590 2024-04-25T00:45:59.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8881ffb640 1 -- 172.21.15.129:0/1655827739 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c0ab600 con 0x7f888415c590 2024-04-25T00:45:59.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8881ffb640 1 -- 172.21.15.129:0/1655827739 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c0ab9d0 con 0x7f888415c590 2024-04-25T00:45:59.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/1655827739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f888415c590 msgr2=0x7f88840cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 --2- 172.21.15.129:0/1655827739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f888415c590 0x7f88840cc790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f886c009870 tx=0x7f886c01c460 comp rx=0 tx=0).stop 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/1655827739 shutdown_connections 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 --2- 172.21.15.129:0/1655827739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f888415c590 0x7f88840cc790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 --2- 172.21.15.129:0/1655827739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8884145960 0x7f8884160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/1655827739 >> 172.21.15.129:0/1655827739 conn(0x7f8884067730 msgr2=0x7f88840b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/1655827739 shutdown_connections 2024-04-25T00:45:59.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/1655827739 wait complete. 2024-04-25T00:45:59.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 Processor -- start 2024-04-25T00:45:59.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- start start 2024-04-25T00:45:59.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88840cccd0 0x7f888413d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 0x7f888413d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f888415c590 0x7f88841322c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88840cf890 con 0x7f8884145960 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f888406d880 con 0x7f88840cccd0 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f888406cc40 con 0x7f888415c590 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 0x7f888413d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 0x7f888413d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42580/0 (socket says 172.21.15.129:42580) 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 -- 172.21.15.129:0/4112237577 learned_addr learned my addr 172.21.15.129:0/4112237577 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88837fe640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f888415c590 0x7f88841322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 -- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f888415c590 msgr2=0x7f88841322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f888415c590 0x7f88841322c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 -- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88840cccd0 msgr2=0x7f888413d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88840cccd0 0x7f888413d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 -- 172.21.15.129:0/4112237577 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8884133060 con 0x7f8884145960 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f88827fc640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 0x7f888413d890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f886c01cd00 tx=0x7f886c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c0b8020 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c0b4610 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8884133340 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c0b0040 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.655+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8884133900 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.659+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8884062e50 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.659+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f886c0bb070 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.659+0000 7f887b7fe640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f88640678f0 0x7f8864069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.659+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f886c11eb40 con 0x7f8884145960 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.659+0000 7f8882ffd640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f88640678f0 0x7f8864069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.659+0000 7f8882ffd640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f88640678f0 0x7f8864069db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f887400a8b0 tx=0x7f8874008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.663+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f886c016110 con 0x7f8884145960 2024-04-25T00:45:59.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8884075320 con 0x7f8884145960 2024-04-25T00:45:59.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f887b7fe640 1 -- 172.21.15.129:0/4112237577 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f886c0ef660 con 0x7f8884145960 2024-04-25T00:45:59.821 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:45:59.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f88640678f0 msgr2=0x7f8864069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f88640678f0 0x7f8864069db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f887400a8b0 tx=0x7f8874008040 comp rx=0 tx=0).stop 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 msgr2=0x7f888413d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 0x7f888413d890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f886c01cd00 tx=0x7f886c019200 comp rx=0 tx=0).stop 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 shutdown_connections 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f88640678f0 0x7f8864069db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f888415c590 0x7f88841322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8884145960 0x7f888413d890 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 --2- 172.21.15.129:0/4112237577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88840cccd0 0x7f888413d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 >> 172.21.15.129:0/4112237577 conn(0x7f8884067730 msgr2=0x7f88841442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 shutdown_connections 2024-04-25T00:45:59.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.819+0000 7f8889425640 1 -- 172.21.15.129:0/4112237577 wait complete. 2024-04-25T00:45:59.839 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:45:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 Processor -- start 2024-04-25T00:45:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 -- start start 2024-04-25T00:45:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7415b0d0 0x7fab740b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fab7406da60 0x7fab7413ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab740cf880 con 0x7fab740cf0a0 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fab740d0490 con 0x7fab7406da60 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab7a9a1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab740cfb60 con 0x7fab7415b0d0 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab78f3e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fab7406da60 0x7fab7413ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab737fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fab740cf0a0 0x7fab7406d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34712/0 (socket says 172.21.15.129:34712) 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab737fe640 1 -- 172.21.15.129:0/1372772267 learned_addr learned my addr 172.21.15.129:0/1372772267 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab73fff640 1 --2- 172.21.15.129:0/1372772267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7415b0d0 0x7fab740b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab78f3e640 1 -- 172.21.15.129:0/1372772267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7415b0d0 msgr2=0x7fab740b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab78f3e640 1 --2- 172.21.15.129:0/1372772267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7415b0d0 0x7fab740b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab78f3e640 1 -- 172.21.15.129:0/1372772267 >> v1:172.21.15.129:6789/0 conn(0x7fab740cf0a0 legacy=0x7fab7406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab72ffd640 1 -- 172.21.15.129:0/1372772267 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399318980 0 0) 0x7fab740cf880 con 0x7fab740cf0a0 2024-04-25T00:45:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.951+0000 7fab78f3e640 1 -- 172.21.15.129:0/1372772267 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab740cfe40 con 0x7fab7406da60 2024-04-25T00:45:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 --2- 172.21.15.129:0/1372772267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7406da60 0x7fab7413ebf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009d90 tx=0x7fab6c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2b3016321a8c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab72ffd640 1 -- 172.21.15.129:0/1372772267 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2070 con 0x7fab7406da60 2024-04-25T00:45:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab72ffd640 1 -- 172.21.15.129:0/1372772267 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c0a5990 con 0x7fab7406da60 2024-04-25T00:45:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab72ffd640 1 -- 172.21.15.129:0/1372772267 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0a5d60 con 0x7fab7406da60 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- 172.21.15.129:0/1372772267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7406da60 msgr2=0x7fab7413ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/1372772267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7406da60 0x7fab7413ebf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009d90 tx=0x7fab6c01c8b0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- 172.21.15.129:0/1372772267 shutdown_connections 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/1372772267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7406da60 0x7fab7413ebf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/1372772267 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7415b0d0 0x7fab740b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- 172.21.15.129:0/1372772267 >> 172.21.15.129:0/1372772267 conn(0x7fab74071ac0 msgr2=0x7fab74071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- 172.21.15.129:0/1372772267 shutdown_connections 2024-04-25T00:45:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- 172.21.15.129:0/1372772267 wait complete. 2024-04-25T00:45:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 Processor -- start 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- start start 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7406da60 0x7fab74155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab740cf0a0 0x7fab741578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 0x7fab740b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab7413fac0 con 0x7fab7415b0d0 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab7413f940 con 0x7fab740cf0a0 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab7a9a1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab7413fc40 con 0x7fab7406da60 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 0x7fab740b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 0x7fab740b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42596/0 (socket says 172.21.15.129:42596) 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 -- 172.21.15.129:0/3282357180 learned_addr learned my addr 172.21.15.129:0/3282357180 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab73fff640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7406da60 0x7fab74155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab737fe640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab740cf0a0 0x7fab741578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 -- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7406da60 msgr2=0x7fab74155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7406da60 0x7fab74155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 -- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab740cf0a0 msgr2=0x7fab741578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab740cf0a0 0x7fab741578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 -- 172.21.15.129:0/3282357180 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74155f70 con 0x7fab7415b0d0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab73fff640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7406da60 0x7fab74155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab78f3e640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 0x7fab740b2f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fab6c01af60 tx=0x7fab6c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.955+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2050 con 0x7fab7415b0d0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c0be070 con 0x7fab7415b0d0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab74157e30 con 0x7fab7415b0d0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0ada50 con 0x7fab7415b0d0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab740ab200 con 0x7fab7415b0d0 2024-04-25T00:45:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab74156420 con 0x7fab7415b0d0 2024-04-25T00:45:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab6c0adbf0 con 0x7fab7415b0d0 2024-04-25T00:45:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab70ff9640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fab540678c0 0x7fab54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fab6c121100 con 0x7fab7415b0d0 2024-04-25T00:45:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab73fff640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fab540678c0 0x7fab54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.959+0000 7fab73fff640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fab540678c0 0x7fab54069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab741f8000 tx=0x7fab64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:45:59.963+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab6c016560 con 0x7fab7415b0d0 2024-04-25T00:46:00.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.115+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab74075320 con 0x7fab7415b0d0 2024-04-25T00:46:00.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.115+0000 7fab70ff9640 1 -- 172.21.15.129:0/3282357180 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fab6c0ab340 con 0x7fab7415b0d0 2024-04-25T00:46:00.117 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:46:00.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.115+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fab540678c0 msgr2=0x7fab54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.115+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fab540678c0 0x7fab54069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab741f8000 tx=0x7fab64008040 comp rx=0 tx=0).stop 2024-04-25T00:46:00.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.115+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 msgr2=0x7fab740b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.115+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 0x7fab740b2f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fab6c01af60 tx=0x7fab6c019650 comp rx=0 tx=0).stop 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 shutdown_connections 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fab540678c0 0x7fab54069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7415b0d0 0x7fab740b2f70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab740cf0a0 0x7fab741578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 --2- 172.21.15.129:0/3282357180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab7406da60 0x7fab74155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 >> 172.21.15.129:0/3282357180 conn(0x7fab74071ac0 msgr2=0x7fab74074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 shutdown_connections 2024-04-25T00:46:00.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.119+0000 7fab7a9a1640 1 -- 172.21.15.129:0/3282357180 wait complete. 2024-04-25T00:46:00.135 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:46:00.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.243+0000 7fbef0e18640 1 Processor -- start 2024-04-25T00:46:00.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.243+0000 7fbef0e18640 1 -- start start 2024-04-25T00:46:00.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbef0e18640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbeec0ce690 0x7fbeec06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbef0e18640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbeec0cea70 0x7fbeec0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbef0e18640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeec0b32e0 con 0x7fbeec0cd050 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbef0e18640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec15acd0 con 0x7fbeec0ce690 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbef0e18640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec0b3ef0 con 0x7fbeec0cea70 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbeec0ce690 0x7fbeec06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbeec0ce690 0x7fbeec06ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42610/0 (socket says 172.21.15.129:42610) 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 -- 172.21.15.129:0/3962289035 learned_addr learned my addr 172.21.15.129:0/3962289035 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 -- 172.21.15.129:0/3962289035 >> v2:172.21.15.196:3300/0 conn(0x7fbeec0cea70 msgr2=0x7fbeec0cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 --2- 172.21.15.129:0/3962289035 >> v2:172.21.15.196:3300/0 conn(0x7fbeec0cea70 0x7fbeec0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 -- 172.21.15.129:0/3962289035 >> v1:172.21.15.196:6789/0 conn(0x7fbeec0cd050 legacy=0x7fbeec0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:00.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 -- 172.21.15.129:0/3962289035 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec0b35c0 con 0x7fbeec0ce690 2024-04-25T00:46:00.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbeea59c640 1 --2- 172.21.15.129:0/3962289035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec06ef40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0afa10 tx=0x7fbedc01eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be2e377aa110ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:00.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbee959a640 1 -- 172.21.15.129:0/3962289035 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0b4070 con 0x7fbeec0ce690 2024-04-25T00:46:00.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbee959a640 1 -- 172.21.15.129:0/3962289035 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc0a7bd0 con 0x7fbeec0ce690 2024-04-25T00:46:00.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.247+0000 7fbee959a640 1 -- 172.21.15.129:0/3962289035 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0af440 con 0x7fbeec0ce690 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3962289035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 msgr2=0x7fbeec06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3962289035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec06ef40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0afa10 tx=0x7fbedc01eb80 comp rx=0 tx=0).stop 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3962289035 shutdown_connections 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3962289035 >> v2:172.21.15.196:3300/0 conn(0x7fbeec0cea70 0x7fbeec0cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3962289035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec06ef40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3962289035 >> 172.21.15.129:0/3962289035 conn(0x7fbeec071ac0 msgr2=0x7fbeec071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3962289035 shutdown_connections 2024-04-25T00:46:00.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3962289035 wait complete. 2024-04-25T00:46:00.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 Processor -- start 2024-04-25T00:46:00.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- start start 2024-04-25T00:46:00.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0cd050 0x7fbeec15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0cea70 0x7fbeec0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec06f600 con 0x7fbeec0ce690 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec06f480 con 0x7fbeec0cea70 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0b2e70 con 0x7fbeec0cd050 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42618/0 (socket says 172.21.15.129:42618) 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 -- 172.21.15.129:0/3980915898 learned_addr learned my addr 172.21.15.129:0/3980915898 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbeea59c640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0cd050 0x7fbeec15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 -- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0cd050 msgr2=0x7fbeec15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0cd050 0x7fbeec15cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 -- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0cea70 msgr2=0x7fbeec0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbeead9d640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0cea70 0x7fbeec0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0cea70 0x7fbeec0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 -- 172.21.15.129:0/3980915898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1518e0 con 0x7fbeec0ce690 2024-04-25T00:46:00.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbee9d9b640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec138880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbee000b580 tx=0x7fbee000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:00.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0014070 con 0x7fbeec0ce690 2024-04-25T00:46:00.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee0010890 con 0x7fbeec0ce690 2024-04-25T00:46:00.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec151bc0 con 0x7fbeec0ce690 2024-04-25T00:46:00.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0010c80 con 0x7fbeec0ce690 2024-04-25T00:46:00.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec139c80 con 0x7fbeec0ce690 2024-04-25T00:46:00.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.251+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec062e50 con 0x7fbeec0ce690 2024-04-25T00:46:00.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.255+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbee0017070 con 0x7fbeec0ce690 2024-04-25T00:46:00.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.255+0000 7fbedaffd640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbecc0678c0 0x7fbecc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.255+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fbee0085b80 con 0x7fbeec0ce690 2024-04-25T00:46:00.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.255+0000 7fbeea59c640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbecc0678c0 0x7fbecc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.255+0000 7fbeea59c640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbecc0678c0 0x7fbecc069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0b1000 tx=0x7fbedc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:00.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.259+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbee0056620 con 0x7fbeec0ce690 2024-04-25T00:46:00.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.407+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbeec0af570 con 0x7fbeec0ce690 2024-04-25T00:46:00.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.407+0000 7fbedaffd640 1 -- 172.21.15.129:0/3980915898 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fbee0010e20 con 0x7fbeec0ce690 2024-04-25T00:46:00.410 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:46:00.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbecc0678c0 msgr2=0x7fbecc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbecc0678c0 0x7fbecc069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0b1000 tx=0x7fbedc008040 comp rx=0 tx=0).stop 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 msgr2=0x7fbeec138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec138880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbee000b580 tx=0x7fbee000ba40 comp rx=0 tx=0).stop 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 shutdown_connections 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbecc0678c0 0x7fbecc069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0cea70 0x7fbeec0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec0ce690 0x7fbeec138880 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 --2- 172.21.15.129:0/3980915898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0cd050 0x7fbeec15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 >> 172.21.15.129:0/3980915898 conn(0x7fbeec071ac0 msgr2=0x7fbeec0cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 shutdown_connections 2024-04-25T00:46:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.411+0000 7fbef0e18640 1 -- 172.21.15.129:0/3980915898 wait complete. 2024-04-25T00:46:00.428 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:46:00.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 Processor -- start 2024-04-25T00:46:00.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 -- start start 2024-04-25T00:46:00.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e380aa0c0 0x7f9e380aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9e380ccb50 0x7f9e380cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e3806d1b0 con 0x7f9e3815bf50 2024-04-25T00:46:00.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3815ca40 con 0x7f9e380ccb50 2024-04-25T00:46:00.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3f0ff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3806d490 con 0x7f9e380aa0c0 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9e380ccb50 0x7f9e380cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9e380ccb50 0x7f9e380cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42628/0 (socket says 172.21.15.129:42628) 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 -- 172.21.15.129:0/6917018 learned_addr learned my addr 172.21.15.129:0/6917018 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e37fff640 1 --2- 172.21.15.129:0/6917018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e380aa0c0 0x7f9e380aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 -- 172.21.15.129:0/6917018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e380aa0c0 msgr2=0x7f9e380aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 --2- 172.21.15.129:0/6917018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e380aa0c0 0x7f9e380aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 -- 172.21.15.129:0/6917018 >> v1:172.21.15.196:6789/0 conn(0x7f9e3815bf50 legacy=0x7f9e3815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:00.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.547+0000 7f9e3d69c640 1 -- 172.21.15.129:0/6917018 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3806d770 con 0x7f9e380ccb50 2024-04-25T00:46:00.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3d69c640 1 --2- 172.21.15.129:0/6917018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380ccb50 0x7f9e380cd000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009e50 tx=0x7f9e3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145850bf12d9ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:00.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e377fe640 1 -- 172.21.15.129:0/6917018 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300a5760 con 0x7f9e380ccb50 2024-04-25T00:46:00.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e377fe640 1 -- 172.21.15.129:0/6917018 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e300a5900 con 0x7f9e380ccb50 2024-04-25T00:46:00.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e377fe640 1 -- 172.21.15.129:0/6917018 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300a5cd0 con 0x7f9e380ccb50 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/6917018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380ccb50 msgr2=0x7f9e380cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/6917018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380ccb50 0x7f9e380cd000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009e50 tx=0x7f9e3001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/6917018 shutdown_connections 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/6917018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380ccb50 0x7f9e380cd000 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/6917018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e380aa0c0 0x7f9e380aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/6917018 >> 172.21.15.129:0/6917018 conn(0x7f9e380b14a0 msgr2=0x7f9e380b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/6917018 shutdown_connections 2024-04-25T00:46:00.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/6917018 wait complete. 2024-04-25T00:46:00.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 Processor -- start 2024-04-25T00:46:00.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- start start 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 0x7f9e3813eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e380ccb50 0x7f9e3814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e3815bf50 0x7f9e3814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3806ea60 con 0x7f9e380aa0c0 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3806cf70 con 0x7f9e380ccb50 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3f0ff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3806ebe0 con 0x7f9e3815bf50 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 0x7f9e3813eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 0x7f9e3813eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42634/0 (socket says 172.21.15.129:42634) 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 -- 172.21.15.129:0/2523059202 learned_addr learned my addr 172.21.15.129:0/2523059202 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:00.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e37fff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e380ccb50 0x7f9e3814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3d69c640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e3815bf50 0x7f9e3814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 -- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e3815bf50 msgr2=0x7f9e3814dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e3815bf50 0x7f9e3814dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 -- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e380ccb50 msgr2=0x7f9e3814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.551+0000 7f9e3ce9b640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e380ccb50 0x7f9e3814a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e3ce9b640 1 -- 172.21.15.129:0/2523059202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e38140db0 con 0x7f9e380aa0c0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e3ce9b640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 0x7f9e3813eb30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00b410 tx=0x7f9e2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c015030 con 0x7f9e380aa0c0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c004020 con 0x7f9e380aa0c0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e38141090 con 0x7f9e380aa0c0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c004410 con 0x7f9e380aa0c0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e38141650 con 0x7f9e380aa0c0 2024-04-25T00:46:00.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e38062e50 con 0x7f9e380aa0c0 2024-04-25T00:46:00.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e2c018070 con 0x7f9e380aa0c0 2024-04-25T00:46:00.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e357fa640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f9e240678c0 0x7f9e24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:00.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f9e2c085930 con 0x7f9e380aa0c0 2024-04-25T00:46:00.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e37fff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f9e240678c0 0x7f9e24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:00.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.555+0000 7f9e37fff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f9e240678c0 0x7f9e24069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e3814af00 tx=0x7f9e28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.559+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e2c0563c0 con 0x7f9e380aa0c0 2024-04-25T00:46:00.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.719+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e38075320 con 0x7f9e380aa0c0 2024-04-25T00:46:00.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.719+0000 7f9e357fa640 1 -- 172.21.15.129:0/2523059202 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f9e2c055cc0 con 0x7f9e380aa0c0 2024-04-25T00:46:00.721 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f9e240678c0 msgr2=0x7f9e24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f9e240678c0 0x7f9e24069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e3814af00 tx=0x7f9e28008040 comp rx=0 tx=0).stop 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 msgr2=0x7f9e3813eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 0x7f9e3813eb30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00b410 tx=0x7f9e2c00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 shutdown_connections 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f9e240678c0 0x7f9e24069d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e3815bf50 0x7f9e3814dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e380ccb50 0x7f9e3814a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 --2- 172.21.15.129:0/2523059202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e380aa0c0 0x7f9e3813eb30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 >> 172.21.15.129:0/2523059202 conn(0x7f9e380b14a0 msgr2=0x7f9e380cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 shutdown_connections 2024-04-25T00:46:00.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:00.723+0000 7f9e3f0ff640 1 -- 172.21.15.129:0/2523059202 wait complete. 2024-04-25T00:46:00.739 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi129.front.sepia.ceph.com... 2024-04-25T00:46:00.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:00.739 DEBUG:teuthology.orchestra.run.smithi129:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T00:46:00.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi129.stderr:ceph-fuse[35095]: fuse finished with error 0 and tester_r 0 2024-04-25T00:46:00.751 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:00.848 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:46:00.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:00.849 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T00:46:00.853 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:46:00.853 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:46:00.860 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:46:00.860 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:46:00.908 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:46:01.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 Processor -- start 2024-04-25T00:46:01.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 -- start start 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f816815c7f0 0x7f81680723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f81680cd190 0x7f816815c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8168072aa0 con 0x7f81680ccdb0 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f8168070690 con 0x7f81680cd190 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816f398640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f81680708e0 con 0x7f816815c7f0 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d935640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f81680cd190 0x7f816815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d134640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f81680ccdb0 0x7f8168158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34716/0 (socket says 172.21.15.129:34716) 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d134640 1 -- 172.21.15.129:0/3478566703 learned_addr learned my addr 172.21.15.129:0/3478566703 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816c933640 1 --2- 172.21.15.129:0/3478566703 >> v2:172.21.15.196:3300/0 conn(0x7f816815c7f0 0x7f81680723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d935640 1 -- 172.21.15.129:0/3478566703 >> v2:172.21.15.196:3300/0 conn(0x7f816815c7f0 msgr2=0x7f81680723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d935640 1 --2- 172.21.15.129:0/3478566703 >> v2:172.21.15.196:3300/0 conn(0x7f816815c7f0 0x7f81680723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d935640 1 -- 172.21.15.129:0/3478566703 >> v1:172.21.15.129:6789/0 conn(0x7f81680ccdb0 legacy=0x7f8168158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.019+0000 7f816d935640 1 -- 172.21.15.129:0/3478566703 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8168070bc0 con 0x7f81680cd190 2024-04-25T00:46:01.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816d935640 1 --2- 172.21.15.129:0/3478566703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f816815c120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8168072d80 tx=0x7f816001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b342cde100bf66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f815bfff640 1 -- 172.21.15.129:0/3478566703 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600a6690 con 0x7f81680cd190 2024-04-25T00:46:01.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f815bfff640 1 -- 172.21.15.129:0/3478566703 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81600a6830 con 0x7f81680cd190 2024-04-25T00:46:01.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f815bfff640 1 -- 172.21.15.129:0/3478566703 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600a6c00 con 0x7f81680cd190 2024-04-25T00:46:01.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/3478566703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 msgr2=0x7f816815c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 --2- 172.21.15.129:0/3478566703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f816815c120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8168072d80 tx=0x7f816001c8a0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/3478566703 shutdown_connections 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 --2- 172.21.15.129:0/3478566703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f816815c120 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 --2- 172.21.15.129:0/3478566703 >> v2:172.21.15.196:3300/0 conn(0x7f816815c7f0 0x7f81680723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/3478566703 >> 172.21.15.129:0/3478566703 conn(0x7f81680b14a0 msgr2=0x7f81680b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/3478566703 shutdown_connections 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/3478566703 wait complete. 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 Processor -- start 2024-04-25T00:46:01.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- start start 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81680ccdb0 0x7f81681f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f8168155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f816815c7f0 0x7f816815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f816806ea10 con 0x7f81680cd190 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f816806e890 con 0x7f81680ccdb0 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f816806eb90 con 0x7f816815c7f0 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f8168155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f8168155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42648/0 (socket says 172.21.15.129:42648) 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 -- 172.21.15.129:0/2807930011 learned_addr learned my addr 172.21.15.129:0/2807930011 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 -- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f816815c7f0 msgr2=0x7f816815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816d935640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f816815c7f0 0x7f816815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f816815c7f0 0x7f816815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 -- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81680ccdb0 msgr2=0x7f81681f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81680ccdb0 0x7f81681f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 -- 172.21.15.129:0/2807930011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81680b08d0 con 0x7f81680cd190 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816c933640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f8168155c00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f815c00d5f0 tx=0x7f815c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c0125f0 con 0x7f81680cd190 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815c019070 con 0x7f81680cd190 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8168159aa0 con 0x7f81680cd190 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c0129b0 con 0x7f81680cd190 2024-04-25T00:46:01.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.023+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816815a060 con 0x7f81680cd190 2024-04-25T00:46:01.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.027+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168062e50 con 0x7f81680cd190 2024-04-25T00:46:01.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.027+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f815c012b50 con 0x7f81680cd190 2024-04-25T00:46:01.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.027+0000 7f8159ffb640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f814c0678c0 0x7f814c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.027+0000 7f816d134640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f814c0678c0 0x7f814c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.027+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f815c086160 con 0x7f81680cd190 2024-04-25T00:46:01.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.027+0000 7f816d134640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f814c0678c0 0x7f814c069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8164000f70 tx=0x7f8164007a90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.031+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f815c011340 con 0x7f81680cd190 2024-04-25T00:46:01.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.179+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8168075320 con 0x7f81680cd190 2024-04-25T00:46:01.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.179+0000 7f8159ffb640 1 -- 172.21.15.129:0/2807930011 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f815c056bf0 con 0x7f81680cd190 2024-04-25T00:46:01.183 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:01.183 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:46:01.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f814c0678c0 msgr2=0x7f814c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f814c0678c0 0x7f814c069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8164000f70 tx=0x7f8164007a90 comp rx=0 tx=0).stop 2024-04-25T00:46:01.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 msgr2=0x7f8168155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f8168155c00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f815c00d5f0 tx=0x7f815c00dab0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 shutdown_connections 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f814c0678c0 0x7f814c069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f816815c7f0 0x7f816815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81680cd190 0x7f8168155c00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 --2- 172.21.15.129:0/2807930011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81680ccdb0 0x7f81681f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 >> 172.21.15.129:0/2807930011 conn(0x7f81680b14a0 msgr2=0x7f81681504f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 shutdown_connections 2024-04-25T00:46:01.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.183+0000 7f816f398640 1 -- 172.21.15.129:0/2807930011 wait complete. 2024-04-25T00:46:01.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:46:01.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:46:01.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.307+0000 7fef96112640 1 Processor -- start 2024-04-25T00:46:01.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.307+0000 7fef96112640 1 -- start start 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef96112640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 0x7fef90165e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef96112640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef900ccb80 con 0x7fef900cf770 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef96112640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef900cce60 con 0x7fef9015bd90 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef96112640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef900cc930 con 0x7fef9015c170 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef87fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fef900cf770 0x7fef9006f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34730/0 (socket says 172.21.15.129:34730) 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef87fff640 1 -- 172.21.15.129:0/1775476887 learned_addr learned my addr 172.21.15.129:0/1775476887 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 0x7fef90165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592214162 0 0) 0x7fef900ccb80 con 0x7fef900cf770 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef7c005740 con 0x7fef900cf770 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491422921 0 0) 0x7fef7c005740 con 0x7fef900cf770 2024-04-25T00:46:01.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 msgr2=0x7fef90165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 0x7fef90165e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 >> v1:172.21.15.196:6789/0 conn(0x7fef9015bd90 legacy=0x7fef900cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900cd140 con 0x7fef900cf770 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fef78001680 con 0x7fef900cf770 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 >> v1:172.21.15.129:6789/0 conn(0x7fef900cf770 legacy=0x7fef9006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c005c00 0x7fef7c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 0x7fef7c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef7c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef900cc930 con 0x7fef7c006f60 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c00ddf0 con 0x7fef7c005c00 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c00dff0 con 0x7fef9015c170 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef7c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef87fff640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 0x7fef7c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8f7fe640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c005c00 0x7fef7c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 -- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 msgr2=0x7fef7c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef9015c170 0x7fef7c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 -- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c005c00 msgr2=0x7fef7c006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c005c00 0x7fef7c006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.311+0000 7fef8ffff640 1 -- 172.21.15.129:0/1775476887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c005740 con 0x7fef7c006f60 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8ffff640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef7c0073e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef88007330 tx=0x7fef8801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cfb4c3d78b4727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880b2070 con 0x7fef7c006f60 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef880a5c10 con 0x7fef7c006f60 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8effd640 1 -- 172.21.15.129:0/1775476887 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88009be0 con 0x7fef7c006f60 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- 172.21.15.129:0/1775476887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 msgr2=0x7fef7c0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 --2- 172.21.15.129:0/1775476887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef7c0073e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef88007330 tx=0x7fef8801ce20 comp rx=0 tx=0).stop 2024-04-25T00:46:01.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8ffff640 1 -- 172.21.15.129:0/1775476887 reap_dead start 2024-04-25T00:46:01.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- 172.21.15.129:0/1775476887 shutdown_connections 2024-04-25T00:46:01.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- 172.21.15.129:0/1775476887 >> 172.21.15.129:0/1775476887 conn(0x7fef90067730 msgr2=0x7fef900b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- 172.21.15.129:0/1775476887 shutdown_connections 2024-04-25T00:46:01.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- 172.21.15.129:0/1775476887 wait complete. 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 Processor -- start 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- start start 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef9013e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef90153d80 0x7fef90152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef90152840 0x7fef9015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef900cd940 con 0x7fef7c006f60 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef900cd4b0 con 0x7fef90153d80 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef96112640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef9006fa80 con 0x7fef90152840 2024-04-25T00:46:01.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef9013e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef9013e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42660/0 (socket says 172.21.15.129:42660) 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 -- 172.21.15.129:0/396510494 learned_addr learned my addr 172.21.15.129:0/396510494 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef87fff640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef90153d80 0x7fef90152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 -- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef90152840 msgr2=0x7fef9015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef90152840 0x7fef9015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 -- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef90153d80 msgr2=0x7fef90152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef90153d80 0x7fef90152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 -- 172.21.15.129:0/396510494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef90152ce0 con 0x7fef7c006f60 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef87fff640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef90153d80 0x7fef90152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.315+0000 7fef8f7fe640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef9013e9a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fef8000e4d0 tx=0x7fef8000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800052e0 con 0x7fef7c006f60 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80018070 con 0x7fef7c006f60 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90152fc0 con 0x7fef7c006f60 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80003820 con 0x7fef7c006f60 2024-04-25T00:46:01.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef90153580 con 0x7fef7c006f60 2024-04-25T00:46:01.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef90062e50 con 0x7fef7c006f60 2024-04-25T00:46:01.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fef800039c0 con 0x7fef7c006f60 2024-04-25T00:46:01.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef8cff9640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fef700678c0 0x7fef70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fef80085a10 con 0x7fef7c006f60 2024-04-25T00:46:01.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef87fff640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fef700678c0 0x7fef70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.319+0000 7fef87fff640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fef700678c0 0x7fef70069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef78004880 tx=0x7fef780047f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.323+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fef800564a0 con 0x7fef7c006f60 2024-04-25T00:46:01.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef90075320 con 0x7fef7c006f60 2024-04-25T00:46:01.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef8cff9640 1 -- 172.21.15.129:0/396510494 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fef80005480 con 0x7fef7c006f60 2024-04-25T00:46:01.472 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:46:01.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fef700678c0 msgr2=0x7fef70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fef700678c0 0x7fef70069d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef78004880 tx=0x7fef780047f0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 msgr2=0x7fef9013e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef9013e9a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fef8000e4d0 tx=0x7fef8000e990 comp rx=0 tx=0).stop 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 shutdown_connections 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fef700678c0 0x7fef70069d80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef90152840 0x7fef9015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef90153d80 0x7fef90152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 --2- 172.21.15.129:0/396510494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef7c006f60 0x7fef9013e9a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 >> 172.21.15.129:0/396510494 conn(0x7fef90067730 msgr2=0x7fef900721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 shutdown_connections 2024-04-25T00:46:01.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.471+0000 7fef96112640 1 -- 172.21.15.129:0/396510494 wait complete. 2024-04-25T00:46:01.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:46:01.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 Processor -- start 2024-04-25T00:46:01.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 -- start start 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1ed4141560 0x7f1ed4072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ed40728a0 0x7f1ed4143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed4071ed0 con 0x7f1ed4141560 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed415ed70 con 0x7f1ed40728a0 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1edbafd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed4144730 con 0x7f1ed41440c0 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1ed4141560 0x7f1ed4072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1ed4141560 0x7f1ed4072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42670/0 (socket says 172.21.15.129:42670) 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 -- 172.21.15.129:0/2740114433 learned_addr learned my addr 172.21.15.129:0/2740114433 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1eda09a640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 -- 172.21.15.129:0/2740114433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 msgr2=0x7f1ed406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 -- 172.21.15.129:0/2740114433 >> v2:172.21.15.196:3300/0 conn(0x7f1ed40728a0 msgr2=0x7f1ed4143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 --2- 172.21.15.129:0/2740114433 >> v2:172.21.15.196:3300/0 conn(0x7f1ed40728a0 0x7f1ed4143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1ed9899640 1 -- 172.21.15.129:0/2740114433 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed4071880 con 0x7f1ed4141560 2024-04-25T00:46:01.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.603+0000 7f1eda09a640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:01.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1ed9899640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4072360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1ed41481c0 tx=0x7f1ed001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54eb3671e8db70a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1ed8897640 1 -- 172.21.15.129:0/2740114433 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00b0070 con 0x7f1ed4141560 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1ed8897640 1 -- 172.21.15.129:0/2740114433 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed00ab600 con 0x7f1ed4141560 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1ed8897640 1 -- 172.21.15.129:0/2740114433 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00ab9d0 con 0x7f1ed4141560 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 -- 172.21.15.129:0/2740114433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 msgr2=0x7f1ed4072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4072360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1ed41481c0 tx=0x7f1ed001c520 comp rx=0 tx=0).stop 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 -- 172.21.15.129:0/2740114433 shutdown_connections 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 --2- 172.21.15.129:0/2740114433 >> v2:172.21.15.196:3300/0 conn(0x7f1ed40728a0 0x7f1ed4143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 --2- 172.21.15.129:0/2740114433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4072360 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 -- 172.21.15.129:0/2740114433 >> 172.21.15.129:0/2740114433 conn(0x7f1ed406b210 msgr2=0x7f1ed406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 -- 172.21.15.129:0/2740114433 shutdown_connections 2024-04-25T00:46:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 -- 172.21.15.129:0/2740114433 wait complete. 2024-04-25T00:46:01.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 Processor -- start 2024-04-25T00:46:01.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.607+0000 7f1edbafd640 1 -- start start 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed40728a0 0x7f1ed4137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed41586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4162420 con 0x7f1ed4141560 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41622a0 con 0x7f1ed40728a0 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed41625a0 con 0x7f1ed41440c0 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42676/0 (socket says 172.21.15.129:42676) 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 -- 172.21.15.129:0/3718174278 learned_addr learned my addr 172.21.15.129:0/3718174278 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eda09a640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed41586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 -- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 msgr2=0x7f1ed41586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9899640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed40728a0 0x7f1ed4137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed41586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 -- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed40728a0 msgr2=0x7f1ed4137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed40728a0 0x7f1ed4137f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 -- 172.21.15.129:0/3718174278 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed4136e60 con 0x7f1ed4141560 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eda09a640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed41586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9098640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4153f70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b550 tx=0x7f1ec400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4014070 con 0x7f1ed4141560 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec4010890 con 0x7f1ed4141560 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed4137140 con 0x7f1ed4141560 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4010c80 con 0x7f1ed4141560 2024-04-25T00:46:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41379e0 con 0x7f1ed4141560 2024-04-25T00:46:01.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed4137be0 con 0x7f1ed4141560 2024-04-25T00:46:01.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ec4017070 con 0x7f1ed4141560 2024-04-25T00:46:01.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eca7fc640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1eb80678f0 0x7f1eb8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f1ec4085b10 con 0x7f1ed4141560 2024-04-25T00:46:01.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9899640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1eb80678f0 0x7f1eb8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.611+0000 7f1ed9899640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1eb80678f0 0x7f1eb8069db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00ad000 tx=0x7f1ed0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.615+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ed4137be0 con 0x7f1ed4141560 2024-04-25T00:46:01.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ed4062e50 con 0x7f1ed4141560 2024-04-25T00:46:01.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1eca7fc640 1 -- 172.21.15.129:0/3718174278 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1ec4010e20 con 0x7f1ed4141560 2024-04-25T00:46:01.772 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1eb80678f0 msgr2=0x7f1eb8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1eb80678f0 0x7f1eb8069db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00ad000 tx=0x7f1ed0008040 comp rx=0 tx=0).stop 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 msgr2=0x7f1ed4153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4153f70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b550 tx=0x7f1ec400ba10 comp rx=0 tx=0).stop 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 shutdown_connections 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f1eb80678f0 0x7f1eb8069db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ed41440c0 0x7f1ed41586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ed4141560 0x7f1ed4153f70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 --2- 172.21.15.129:0/3718174278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed40728a0 0x7f1ed4137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 >> 172.21.15.129:0/3718174278 conn(0x7f1ed406b210 msgr2=0x7f1ed4140820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 shutdown_connections 2024-04-25T00:46:01.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.771+0000 7f1edbafd640 1 -- 172.21.15.129:0/3718174278 wait complete. 2024-04-25T00:46:01.791 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:46:01.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 Processor -- start 2024-04-25T00:46:01.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 -- start start 2024-04-25T00:46:01.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1804141010 0x7f18040d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1804144570 0x7f1804143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18040d29c0 con 0x7f18040d2540 2024-04-25T00:46:01.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f18040d2ca0 con 0x7f1804144570 2024-04-25T00:46:01.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1809066640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f18040d2f80 con 0x7f1804141010 2024-04-25T00:46:01.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1804144570 0x7f1804143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1804144570 0x7f1804143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42682/0 (socket says 172.21.15.129:42682) 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 -- 172.21.15.129:0/454381680 learned_addr learned my addr 172.21.15.129:0/454381680 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f180359e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f18040d2540 0x7f1804161db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34732/0 (socket says 172.21.15.129:34732) 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f1802d9d640 1 --2- 172.21.15.129:0/454381680 >> v2:172.21.15.196:3300/0 conn(0x7f1804141010 0x7f18040d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 -- 172.21.15.129:0/454381680 >> v2:172.21.15.196:3300/0 conn(0x7f1804141010 msgr2=0x7f18040d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 --2- 172.21.15.129:0/454381680 >> v2:172.21.15.196:3300/0 conn(0x7f1804141010 0x7f18040d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 -- 172.21.15.129:0/454381680 >> v1:172.21.15.129:6789/0 conn(0x7f18040d2540 legacy=0x7f1804161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f180259c640 1 -- 172.21.15.129:0/454381680 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993540480 0 0) 0x7f18040d29c0 con 0x7f18040d2540 2024-04-25T00:46:01.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.911+0000 7f17fbfff640 1 -- 172.21.15.129:0/454381680 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040d3260 con 0x7f1804144570 2024-04-25T00:46:01.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f17fbfff640 1 --2- 172.21.15.129:0/454381680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804144570 0x7f1804143940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18040d3c20 tx=0x7f17ec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad31e9cc9b39abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f180259c640 1 -- 172.21.15.129:0/454381680 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0b0070 con 0x7f1804144570 2024-04-25T00:46:01.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f180259c640 1 -- 172.21.15.129:0/454381680 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ec0ab600 con 0x7f1804144570 2024-04-25T00:46:01.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f180259c640 1 -- 172.21.15.129:0/454381680 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0ab9d0 con 0x7f1804144570 2024-04-25T00:46:01.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/454381680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804144570 msgr2=0x7f1804143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 --2- 172.21.15.129:0/454381680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804144570 0x7f1804143940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18040d3c20 tx=0x7f17ec01c460 comp rx=0 tx=0).stop 2024-04-25T00:46:01.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/454381680 shutdown_connections 2024-04-25T00:46:01.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 --2- 172.21.15.129:0/454381680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804144570 0x7f1804143940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 --2- 172.21.15.129:0/454381680 >> v2:172.21.15.196:3300/0 conn(0x7f1804141010 0x7f18040d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/454381680 >> 172.21.15.129:0/454381680 conn(0x7f1804072be0 msgr2=0x7f18040d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:01.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/454381680 shutdown_connections 2024-04-25T00:46:01.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/454381680 wait complete. 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 Processor -- start 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- start start 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 0x7f18041538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1804141010 0x7f18040ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804144570 0x7f18040abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1804141600 con 0x7f18040d2540 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1804142ad0 con 0x7f1804144570 2024-04-25T00:46:01.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f180416b150 con 0x7f1804141010 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 0x7f18041538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 0x7f18041538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42690/0 (socket says 172.21.15.129:42690) 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 -- 172.21.15.129:0/2014111761 learned_addr learned my addr 172.21.15.129:0/2014111761 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f17fbfff640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1804141010 0x7f18040ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f180359e640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804144570 0x7f18040abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 -- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1804141010 msgr2=0x7f18040ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1804141010 0x7f18040ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 -- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804144570 msgr2=0x7f18040abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804144570 0x7f18040abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 -- 172.21.15.129:0/2014111761 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180413f8c0 con 0x7f18040d2540 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f180359e640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804144570 0x7f18040abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f17fbfff640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1804141010 0x7f18040ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1802d9d640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 0x7f18041538f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f17f400d800 tx=0x7f17f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f400b6c0 con 0x7f18040d2540 2024-04-25T00:46:01.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f4019070 con 0x7f18040d2540 2024-04-25T00:46:01.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180413fba0 con 0x7f18040d2540 2024-04-25T00:46:01.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f400ba60 con 0x7f18040d2540 2024-04-25T00:46:01.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.915+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1804140160 con 0x7f18040d2540 2024-04-25T00:46:01.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.919+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180415a410 con 0x7f18040d2540 2024-04-25T00:46:01.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.919+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f17f400bc00 con 0x7f18040d2540 2024-04-25T00:46:01.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.919+0000 7f17fb7fe640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17e40678c0 0x7f17e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:01.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.919+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f17f4086170 con 0x7f18040d2540 2024-04-25T00:46:01.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.919+0000 7f17fbfff640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17e40678c0 0x7f17e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:01.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.919+0000 7f17fbfff640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17e40678c0 0x7f17e4069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0ad000 tx=0x7f17ec0a6000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:01.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:01.923+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f17f4056c90 con 0x7f18040d2540 2024-04-25T00:46:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18041403a0 con 0x7f18040d2540 2024-04-25T00:46:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f17fb7fe640 1 -- 172.21.15.129:0/2014111761 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f17f4056590 con 0x7f18040d2540 2024-04-25T00:46:02.085 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:46:02.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17e40678c0 msgr2=0x7f17e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17e40678c0 0x7f17e4069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0ad000 tx=0x7f17ec0a6000 comp rx=0 tx=0).stop 2024-04-25T00:46:02.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 msgr2=0x7f18041538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 0x7f18041538f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f17f400d800 tx=0x7f17f400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 shutdown_connections 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f17e40678c0 0x7f17e4069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1804141010 0x7f18040ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804144570 0x7f18040abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 --2- 172.21.15.129:0/2014111761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18040d2540 0x7f18041538f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 >> 172.21.15.129:0/2014111761 conn(0x7f1804072be0 msgr2=0x7f18040d4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 shutdown_connections 2024-04-25T00:46:02.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.083+0000 7f1809066640 1 -- 172.21.15.129:0/2014111761 wait complete. 2024-04-25T00:46:02.104 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:46:02.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 Processor -- start 2024-04-25T00:46:02.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 -- start start 2024-04-25T00:46:02.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f43d415bf50 0x7f43d415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f43d40cd260 0x7f43d406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d406d1b0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f43d406d490 con 0x7f43d40cd260 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43daf06640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f43d406d770 con 0x7f43d415bf50 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f43d40cd260 0x7f43d406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f43d40cd260 0x7f43d406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42702/0 (socket says 172.21.15.129:42702) 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 -- 172.21.15.129:0/2402064329 learned_addr learned my addr 172.21.15.129:0/2402064329 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d8ca2640 1 --2- 172.21.15.129:0/2402064329 >> v2:172.21.15.196:3300/0 conn(0x7f43d415bf50 0x7f43d415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 -- 172.21.15.129:0/2402064329 >> v2:172.21.15.196:3300/0 conn(0x7f43d415bf50 msgr2=0x7f43d415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 --2- 172.21.15.129:0/2402064329 >> v2:172.21.15.196:3300/0 conn(0x7f43d415bf50 0x7f43d415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 -- 172.21.15.129:0/2402064329 >> v1:172.21.15.196:6789/0 conn(0x7f43d40aa0c0 legacy=0x7f43d40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:02.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d94a3640 1 -- 172.21.15.129:0/2402064329 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d406da50 con 0x7f43d40cd260 2024-04-25T00:46:02.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.211+0000 7f43d8ca2640 1 --2- 172.21.15.129:0/2402064329 >> v2:172.21.15.196:3300/0 conn(0x7f43d415bf50 0x7f43d415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:02.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43d94a3640 1 --2- 172.21.15.129:0/2402064329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40cd260 0x7f43d406e250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f43d4069260 tx=0x7f43cc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b90f8798719126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43d37fe640 1 -- 172.21.15.129:0/2402064329 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0a5860 con 0x7f43d40cd260 2024-04-25T00:46:02.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43d37fe640 1 -- 172.21.15.129:0/2402064329 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc0a5a00 con 0x7f43d40cd260 2024-04-25T00:46:02.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43d37fe640 1 -- 172.21.15.129:0/2402064329 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0a5dd0 con 0x7f43d40cd260 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 -- 172.21.15.129:0/2402064329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40cd260 msgr2=0x7f43d406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 --2- 172.21.15.129:0/2402064329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40cd260 0x7f43d406e250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f43d4069260 tx=0x7f43cc01c980 comp rx=0 tx=0).stop 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 -- 172.21.15.129:0/2402064329 shutdown_connections 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 --2- 172.21.15.129:0/2402064329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40cd260 0x7f43d406e250 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 --2- 172.21.15.129:0/2402064329 >> v2:172.21.15.196:3300/0 conn(0x7f43d415bf50 0x7f43d415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 -- 172.21.15.129:0/2402064329 >> 172.21.15.129:0/2402064329 conn(0x7f43d40b14a0 msgr2=0x7f43d40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 -- 172.21.15.129:0/2402064329 shutdown_connections 2024-04-25T00:46:02.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 -- 172.21.15.129:0/2402064329 wait complete. 2024-04-25T00:46:02.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 Processor -- start 2024-04-25T00:46:02.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.215+0000 7f43daf06640 1 -- start start 2024-04-25T00:46:02.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 0x7f43d4133b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43d40cd260 0x7f43d4141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d415bf50 0x7f43d4141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41691b0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4169030 con 0x7f43d415bf50 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f43d4169330 con 0x7f43d40cd260 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 0x7f43d4133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 0x7f43d4133b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42704/0 (socket says 172.21.15.129:42704) 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 -- 172.21.15.129:0/1532464451 learned_addr learned my addr 172.21.15.129:0/1532464451 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d3fff640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43d40cd260 0x7f43d4141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 -- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43d40cd260 msgr2=0x7f43d4141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43d40cd260 0x7f43d4141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 -- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d415bf50 msgr2=0x7f43d4141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d415bf50 0x7f43d4141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 -- 172.21.15.129:0/1532464451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d4142d10 con 0x7f43d40aa0c0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d8ca2640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 0x7f43d4133b80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c820 tx=0x7f43c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8015070 con 0x7f43d40aa0c0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c80079d0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8007de0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d4142ff0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d41435b0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d4062e50 con 0x7f43d40aa0c0 2024-04-25T00:46:02.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f43c8018070 con 0x7f43d40aa0c0 2024-04-25T00:46:02.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d17fa640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f43b80678c0 0x7f43b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f43c8085c10 con 0x7f43d40aa0c0 2024-04-25T00:46:02.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.219+0000 7f43d3fff640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f43b80678c0 0x7f43b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.223+0000 7f43d3fff640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f43b80678c0 0x7f43b8069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f43d41f8e50 tx=0x7f43c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.223+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f43c80566a0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.383+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43d4075320 con 0x7f43d40aa0c0 2024-04-25T00:46:02.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.383+0000 7f43d17fa640 1 -- 172.21.15.129:0/1532464451 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f43c8055fa0 con 0x7f43d40aa0c0 2024-04-25T00:46:02.386 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f43b80678c0 msgr2=0x7f43b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f43b80678c0 0x7f43b8069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f43d41f8e50 tx=0x7f43c0009040 comp rx=0 tx=0).stop 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 msgr2=0x7f43d4133b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 0x7f43d4133b80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c820 tx=0x7f43c800cce0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 shutdown_connections 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f43b80678c0 0x7f43b8069d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43d40cd260 0x7f43d4141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43d415bf50 0x7f43d4141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 --2- 172.21.15.129:0/1532464451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43d40aa0c0 0x7f43d4133b80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 >> 172.21.15.129:0/1532464451 conn(0x7f43d40b14a0 msgr2=0x7f43d4134320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:02.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 shutdown_connections 2024-04-25T00:46:02.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.387+0000 7f43daf06640 1 -- 172.21.15.129:0/1532464451 wait complete. 2024-04-25T00:46:02.403 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:46:02.404 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:46:02.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:46:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.523+0000 7ff2a92ff640 1 Processor -- start 2024-04-25T00:46:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.523+0000 7ff2a92ff640 1 -- start start 2024-04-25T00:46:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff2a415bf50 0x7ff2a415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff2a40cd260 0x7ff2a406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a406d1b0 con 0x7ff2a40aa0c0 2024-04-25T00:46:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a406d490 con 0x7ff2a40cd260 2024-04-25T00:46:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a406d770 con 0x7ff2a415bf50 2024-04-25T00:46:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff2a40cd260 0x7ff2a406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff2a40cd260 0x7ff2a406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42714/0 (socket says 172.21.15.129:42714) 2024-04-25T00:46:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1595823204 learned_addr learned my addr 172.21.15.129:0/1595823204 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:02.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a2ffd640 1 --2- 172.21.15.129:0/1595823204 >> v2:172.21.15.196:3300/0 conn(0x7ff2a415bf50 0x7ff2a415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1595823204 >> v2:172.21.15.196:3300/0 conn(0x7ff2a415bf50 msgr2=0x7ff2a415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:02.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 --2- 172.21.15.129:0/1595823204 >> v2:172.21.15.196:3300/0 conn(0x7ff2a415bf50 0x7ff2a415c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1595823204 >> v1:172.21.15.196:6789/0 conn(0x7ff2a40aa0c0 legacy=0x7ff2a40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:02.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1595823204 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a406da50 con 0x7ff2a40cd260 2024-04-25T00:46:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a37fe640 1 --2- 172.21.15.129:0/1595823204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a40cd260 0x7ff2a406e250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4069260 tx=0x7ff29c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdce040e69a3fb49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a27fc640 1 -- 172.21.15.129:0/1595823204 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0a5860 con 0x7ff2a40cd260 2024-04-25T00:46:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a27fc640 1 -- 172.21.15.129:0/1595823204 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff29c0a5a00 con 0x7ff2a40cd260 2024-04-25T00:46:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a27fc640 1 -- 172.21.15.129:0/1595823204 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0a5dd0 con 0x7ff2a40cd260 2024-04-25T00:46:02.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1595823204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a40cd260 msgr2=0x7ff2a406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.527+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1595823204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a40cd260 0x7ff2a406e250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4069260 tx=0x7ff29c01c980 comp rx=0 tx=0).stop 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1595823204 shutdown_connections 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1595823204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a40cd260 0x7ff2a406e250 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1595823204 >> v2:172.21.15.196:3300/0 conn(0x7ff2a415bf50 0x7ff2a415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1595823204 >> 172.21.15.129:0/1595823204 conn(0x7ff2a40b14a0 msgr2=0x7ff2a40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1595823204 shutdown_connections 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1595823204 wait complete. 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 Processor -- start 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- start start 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2a40aa0c0 0x7ff2a4147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a40cd260 0x7ff2a40b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 0x7ff2a40b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41691b0 con 0x7ff2a415bf50 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4169030 con 0x7ff2a40cd260 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a4169330 con 0x7ff2a40aa0c0 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 0x7ff2a40b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 0x7ff2a40b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42730/0 (socket says 172.21.15.129:42730) 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1419187815 learned_addr learned my addr 172.21.15.129:0/1419187815 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a2ffd640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2a40aa0c0 0x7ff2a4147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff29bfff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a40cd260 0x7ff2a40b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2a40aa0c0 msgr2=0x7ff2a4147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2a40aa0c0 0x7ff2a4147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a40cd260 msgr2=0x7ff2a40b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a40cd260 0x7ff2a40b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 -- 172.21.15.129:0/1419187815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a4157500 con 0x7ff2a415bf50 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff29bfff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a40cd260 0x7ff2a40b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a37fe640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 0x7ff2a40b06e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff29c01a5b0 tx=0x7ff29c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0bc020 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff29c0b8880 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a41577e0 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0b8c70 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a4157ac0 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.531+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a4062e50 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.535+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff29c0bf020 con 0x7ff2a415bf50 2024-04-25T00:46:02.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.535+0000 7ff29b7fe640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff2840678c0 0x7ff284069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.535+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7ff29c120ef0 con 0x7ff2a415bf50 2024-04-25T00:46:02.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.535+0000 7ff2a2ffd640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff2840678c0 0x7ff284069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.535+0000 7ff2a2ffd640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff2840678c0 0x7ff284069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff29400a8d0 tx=0x7ff294008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.539+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff29c0c6030 con 0x7ff2a415bf50 2024-04-25T00:46:02.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.703+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2a4159230 con 0x7ff2a415bf50 2024-04-25T00:46:02.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.703+0000 7ff29b7fe640 1 -- 172.21.15.129:0/1419187815 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7ff29c0165a0 con 0x7ff2a415bf50 2024-04-25T00:46:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:02.706 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff2840678c0 msgr2=0x7ff284069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff2840678c0 0x7ff284069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff29400a8d0 tx=0x7ff294008040 comp rx=0 tx=0).stop 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 msgr2=0x7ff2a40b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 0x7ff2a40b06e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff29c01a5b0 tx=0x7ff29c019720 comp rx=0 tx=0).stop 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 shutdown_connections 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff2840678c0 0x7ff284069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a415bf50 0x7ff2a40b06e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2a40cd260 0x7ff2a40b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 --2- 172.21.15.129:0/1419187815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2a40aa0c0 0x7ff2a4147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 >> 172.21.15.129:0/1419187815 conn(0x7ff2a40b14a0 msgr2=0x7ff2a4148d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:02.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 shutdown_connections 2024-04-25T00:46:02.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.707+0000 7ff2a92ff640 1 -- 172.21.15.129:0/1419187815 wait complete. 2024-04-25T00:46:02.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:46:02.725 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:46:02.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 Processor -- start 2024-04-25T00:46:02.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 -- start start 2024-04-25T00:46:02.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78f815bd90 0x7f78f80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f78f815c170 0x7f78f80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f80ccb80 con 0x7f78f806f370 2024-04-25T00:46:02.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f78f80cc930 con 0x7f78f815c170 2024-04-25T00:46:02.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78fe6e3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f78f80cd4b0 con 0x7f78f815bd90 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78f77fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f78f815c170 0x7f78f80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78f77fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f78f815c170 0x7f78f80cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42746/0 (socket says 172.21.15.129:42746) 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78f77fe640 1 -- 172.21.15.129:0/2009173573 learned_addr learned my addr 172.21.15.129:0/2009173573 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.847+0000 7f78f7fff640 1 --2- 172.21.15.129:0/2009173573 >> v2:172.21.15.196:3300/0 conn(0x7f78f815bd90 0x7f78f80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f77fe640 1 -- 172.21.15.129:0/2009173573 >> v2:172.21.15.196:3300/0 conn(0x7f78f815bd90 msgr2=0x7f78f80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f77fe640 1 --2- 172.21.15.129:0/2009173573 >> v2:172.21.15.196:3300/0 conn(0x7f78f815bd90 0x7f78f80cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f77fe640 1 -- 172.21.15.129:0/2009173573 >> v1:172.21.15.196:6789/0 conn(0x7f78f806f370 legacy=0x7f78f806f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f77fe640 1 -- 172.21.15.129:0/2009173573 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80cce60 con 0x7f78f815c170 2024-04-25T00:46:02.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f7fff640 1 --2- 172.21.15.129:0/2009173573 >> v2:172.21.15.196:3300/0 conn(0x7f78f815bd90 0x7f78f80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:02.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f77fe640 1 --2- 172.21.15.129:0/2009173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f80cf5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f78f80cdcc0 tx=0x7f78ec01eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3842c80e6fadfcb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f6ffd640 1 -- 172.21.15.129:0/2009173573 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0b4070 con 0x7f78f815c170 2024-04-25T00:46:02.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f6ffd640 1 -- 172.21.15.129:0/2009173573 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78ec0a7980 con 0x7f78f815c170 2024-04-25T00:46:02.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78f6ffd640 1 -- 172.21.15.129:0/2009173573 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0a7d50 con 0x7f78f815c170 2024-04-25T00:46:02.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2009173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 msgr2=0x7f78f80cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2009173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f80cf5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f78f80cdcc0 tx=0x7f78ec01eb20 comp rx=0 tx=0).stop 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2009173573 shutdown_connections 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2009173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f80cf5a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2009173573 >> v2:172.21.15.196:3300/0 conn(0x7f78f815bd90 0x7f78f80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2009173573 >> 172.21.15.129:0/2009173573 conn(0x7f78f8067730 msgr2=0x7f78f80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2009173573 shutdown_connections 2024-04-25T00:46:02.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.851+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2009173573 wait complete. 2024-04-25T00:46:02.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 Processor -- start 2024-04-25T00:46:02.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- start start 2024-04-25T00:46:02.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f806f370 0x7f78f81470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f815bd90 0x7f78f814c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f81477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80ceb70 con 0x7f78f815c170 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80ce9f0 con 0x7f78f815bd90 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78f80cecf0 con 0x7f78f806f370 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f81477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f81477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42760/0 (socket says 172.21.15.129:42760) 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 -- 172.21.15.129:0/2564921194 learned_addr learned my addr 172.21.15.129:0/2564921194 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f7fff640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f806f370 0x7f78f81470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f77fe640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f815bd90 0x7f78f814c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 -- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f806f370 msgr2=0x7f78f81470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f806f370 0x7f78f81470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 -- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f815bd90 msgr2=0x7f78f814c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f815bd90 0x7f78f814c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 -- 172.21.15.129:0/2564921194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f8152af0 con 0x7f78f815c170 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fcc80640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f81477c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78f000b970 tx=0x7f78f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0019070 con 0x7f78f815c170 2024-04-25T00:46:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f000cb40 con 0x7f78f815c170 2024-04-25T00:46:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f8152dd0 con 0x7f78f815c170 2024-04-25T00:46:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0014440 con 0x7f78f815c170 2024-04-25T00:46:02.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f8153390 con 0x7f78f815c170 2024-04-25T00:46:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8062e50 con 0x7f78f815c170 2024-04-25T00:46:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f78f001c070 con 0x7f78f815c170 2024-04-25T00:46:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f4ff9640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78e40678c0 0x7f78e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f78f0085960 con 0x7f78f815c170 2024-04-25T00:46:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.855+0000 7f78f7fff640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78e40678c0 0x7f78e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:02.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.859+0000 7f78f7fff640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78e40678c0 0x7f78e4069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78e8006fd0 tx=0x7f78e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:02.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:02.863+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78f0056480 con 0x7f78f815c170 2024-04-25T00:46:03.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.015+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78f8165c40 con 0x7f78f815c170 2024-04-25T00:46:03.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.015+0000 7f78f4ff9640 1 -- 172.21.15.129:0/2564921194 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f78f0055d80 con 0x7f78f815c170 2024-04-25T00:46:03.018 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:46:03.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78e40678c0 msgr2=0x7f78e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78e40678c0 0x7f78e4069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f78e8006fd0 tx=0x7f78e8008040 comp rx=0 tx=0).stop 2024-04-25T00:46:03.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 msgr2=0x7f78f81477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f81477c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78f000b970 tx=0x7f78f000be30 comp rx=0 tx=0).stop 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 shutdown_connections 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f78e40678c0 0x7f78e4069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78f815c170 0x7f78f81477c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f815bd90 0x7f78f814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 --2- 172.21.15.129:0/2564921194 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f806f370 0x7f78f81470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 >> 172.21.15.129:0/2564921194 conn(0x7f78f8067730 msgr2=0x7f78f814e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 shutdown_connections 2024-04-25T00:46:03.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.019+0000 7f78fe6e3640 1 -- 172.21.15.129:0/2564921194 wait complete. 2024-04-25T00:46:03.036 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:46:03.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.143+0000 7fb72f76b640 1 Processor -- start 2024-04-25T00:46:03.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.143+0000 7fb72f76b640 1 -- start start 2024-04-25T00:46:03.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72f76b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb7280ccea0 0x7fb728166f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72f76b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72806dbf0 con 0x7fb728145960 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72f76b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72806f090 con 0x7fb72815c590 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72f76b640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fb72806cfe0 con 0x7fb7280ccea0 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72d507640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fb728145960 0x7fb728160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46942/0 (socket says 172.21.15.129:46942) 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72d507640 1 -- 172.21.15.129:0/54495117 learned_addr learned my addr 172.21.15.129:0/54495117 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72dd08640 1 --2- 172.21.15.129:0/54495117 >> v2:172.21.15.129:3300/0 conn(0x7fb7280ccea0 0x7fb728166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb71ffff640 1 -- 172.21.15.129:0/54495117 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485797046 0 0) 0x7fb72806dbf0 con 0x7fb728145960 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb71ffff640 1 -- 172.21.15.129:0/54495117 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb714003530 con 0x7fb728145960 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72dd08640 1 -- 172.21.15.129:0/54495117 >> v1:172.21.15.196:6789/0 conn(0x7fb72815c590 legacy=0x7fb7280cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72dd08640 1 -- 172.21.15.129:0/54495117 >> v1:172.21.15.129:6789/0 conn(0x7fb728145960 legacy=0x7fb728160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72dd08640 1 -- 172.21.15.129:0/54495117 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72806d2c0 con 0x7fb7280ccea0 2024-04-25T00:46:03.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb72dd08640 1 --2- 172.21.15.129:0/54495117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb728166f10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb720009d90 tx=0x7fb72001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7d4f5bd9d5b89b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb71ffff640 1 -- 172.21.15.129:0/54495117 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200b2070 con 0x7fb7280ccea0 2024-04-25T00:46:03.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb71ffff640 1 -- 172.21.15.129:0/54495117 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7200a5970 con 0x7fb7280ccea0 2024-04-25T00:46:03.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.147+0000 7fb71ffff640 1 -- 172.21.15.129:0/54495117 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200a5d40 con 0x7fb7280ccea0 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/54495117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 msgr2=0x7fb728166f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 --2- 172.21.15.129:0/54495117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb728166f10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb720009d90 tx=0x7fb72001c8b0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/54495117 shutdown_connections 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 --2- 172.21.15.129:0/54495117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb728166f10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/54495117 >> 172.21.15.129:0/54495117 conn(0x7fb728067730 msgr2=0x7fb7280b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/54495117 shutdown_connections 2024-04-25T00:46:03.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/54495117 wait complete. 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 Processor -- start 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- start start 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb72815f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb728145960 0x7fb72813c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72815c590 0x7fb7281fa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb72806cdc0 con 0x7fb7280ccea0 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb72806cc40 con 0x7fb72815c590 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb728145d40 con 0x7fb728145960 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb72815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb72815f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52244/0 (socket says 172.21.15.129:52244) 2024-04-25T00:46:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 -- 172.21.15.129:0/3470273115 learned_addr learned my addr 172.21.15.129:0/3470273115 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72dd08640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72815c590 0x7fb7281fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 -- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb728145960 msgr2=0x7fb72813c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb728145960 0x7fb72813c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 -- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72815c590 msgr2=0x7fb7281fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72815c590 0x7fb7281fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 -- 172.21.15.129:0/3470273115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72815f770 con 0x7fb7280ccea0 2024-04-25T00:46:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72dd08640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72815c590 0x7fb7281fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72d507640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb72815f230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c4d0 tx=0x7fb72400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724017070 con 0x7fb7280ccea0 2024-04-25T00:46:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb724005cf0 con 0x7fb7280ccea0 2024-04-25T00:46:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb728159d90 con 0x7fb7280ccea0 2024-04-25T00:46:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724007270 con 0x7fb7280ccea0 2024-04-25T00:46:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72815a350 con 0x7fb7280ccea0 2024-04-25T00:46:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.151+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb728062e50 con 0x7fb7280ccea0 2024-04-25T00:46:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.155+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb72401a070 con 0x7fb7280ccea0 2024-04-25T00:46:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.155+0000 7fb71dffb640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb70c0678c0 0x7fb70c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.155+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb724084fc0 con 0x7fb7280ccea0 2024-04-25T00:46:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.155+0000 7fb72cd06640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb70c0678c0 0x7fb70c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.155+0000 7fb72cd06640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb70c0678c0 0x7fb70c069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb718000f70 tx=0x7fb7180079e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.159+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb72815f770 con 0x7fb7280ccea0 2024-04-25T00:46:03.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.303+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb728075320 con 0x7fb7280ccea0 2024-04-25T00:46:03.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.303+0000 7fb71dffb640 1 -- 172.21.15.129:0/3470273115 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fb724055400 con 0x7fb7280ccea0 2024-04-25T00:46:03.308 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:46:03.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb70c0678c0 msgr2=0x7fb70c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb70c0678c0 0x7fb70c069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb718000f70 tx=0x7fb7180079e0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 msgr2=0x7fb72815f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb72815f230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c4d0 tx=0x7fb72400c990 comp rx=0 tx=0).stop 2024-04-25T00:46:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 shutdown_connections 2024-04-25T00:46:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb70c0678c0 0x7fb70c069d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb728145960 0x7fb72813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72815c590 0x7fb7281fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 --2- 172.21.15.129:0/3470273115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7280ccea0 0x7fb72815f230 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 >> 172.21.15.129:0/3470273115 conn(0x7fb728067730 msgr2=0x7fb72806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 shutdown_connections 2024-04-25T00:46:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.307+0000 7fb72f76b640 1 -- 172.21.15.129:0/3470273115 wait complete. 2024-04-25T00:46:03.325 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:46:03.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 Processor -- start 2024-04-25T00:46:03.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 -- start start 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15c590 0x7f839c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f839c0cccd0 0x7f839c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c06dbf0 con 0x7f839c145960 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f839c06cfe0 con 0x7f839c0cccd0 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a2bf8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f839c06d2c0 con 0x7f839c15c590 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f839c0cccd0 0x7f839c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f839c0cccd0 0x7f839c06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52246/0 (socket says 172.21.15.129:52246) 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 -- 172.21.15.129:0/979128869 learned_addr learned my addr 172.21.15.129:0/979128869 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f839bfff640 1 --2- 172.21.15.129:0/979128869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15c590 0x7f839c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 -- 172.21.15.129:0/979128869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15c590 msgr2=0x7f839c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 --2- 172.21.15.129:0/979128869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15c590 0x7f839c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 -- 172.21.15.129:0/979128869 >> v1:172.21.15.196:6789/0 conn(0x7f839c145960 legacy=0x7f839c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 -- 172.21.15.129:0/979128869 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c06d5a0 con 0x7f839c0cccd0 2024-04-25T00:46:03.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f83a1195640 1 --2- 172.21.15.129:0/979128869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c06e080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8394009e50 tx=0x7f839401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d37920981cbc4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f839b7fe640 1 -- 172.21.15.129:0/979128869 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940a56f0 con 0x7f839c0cccd0 2024-04-25T00:46:03.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f839b7fe640 1 -- 172.21.15.129:0/979128869 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83940a5890 con 0x7f839c0cccd0 2024-04-25T00:46:03.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.447+0000 7f839b7fe640 1 -- 172.21.15.129:0/979128869 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940a5c60 con 0x7f839c0cccd0 2024-04-25T00:46:03.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/979128869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 msgr2=0x7f839c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/979128869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c06e080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8394009e50 tx=0x7f839401c960 comp rx=0 tx=0).stop 2024-04-25T00:46:03.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/979128869 shutdown_connections 2024-04-25T00:46:03.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/979128869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c06e080 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/979128869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c15c590 0x7f839c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/979128869 >> 172.21.15.129:0/979128869 conn(0x7f839c067730 msgr2=0x7f839c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/979128869 shutdown_connections 2024-04-25T00:46:03.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/979128869 wait complete. 2024-04-25T00:46:03.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 Processor -- start 2024-04-25T00:46:03.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- start start 2024-04-25T00:46:03.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c145960 0x7f839c15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c15c590 0x7f839c0af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f839c145ec0 con 0x7f839c0cccd0 2024-04-25T00:46:03.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f839c145d40 con 0x7f839c15c590 2024-04-25T00:46:03.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f839c06edd0 con 0x7f839c145960 2024-04-25T00:46:03.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52258/0 (socket says 172.21.15.129:52258) 2024-04-25T00:46:03.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 -- 172.21.15.129:0/2667102164 learned_addr learned my addr 172.21.15.129:0/2667102164 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:03.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f839bfff640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c145960 0x7f839c15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a1195640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c15c590 0x7f839c0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 -- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c145960 msgr2=0x7f839c15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c145960 0x7f839c15b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 -- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c15c590 msgr2=0x7f839c0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c15c590 0x7f839c0af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 -- 172.21.15.129:0/2667102164 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c135590 con 0x7f839c0cccd0 2024-04-25T00:46:03.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a0994640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c15a180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c480 tx=0x7f838c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c012830 con 0x7f839c0cccd0 2024-04-25T00:46:03.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c017070 con 0x7f839c0cccd0 2024-04-25T00:46:03.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c135870 con 0x7f839c0cccd0 2024-04-25T00:46:03.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c012c30 con 0x7f839c0cccd0 2024-04-25T00:46:03.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.451+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c135e30 con 0x7f839c0cccd0 2024-04-25T00:46:03.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.455+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c062e50 con 0x7f839c0cccd0 2024-04-25T00:46:03.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.455+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f838c01c460 con 0x7f839c0cccd0 2024-04-25T00:46:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.455+0000 7f83997fa640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f83880678c0 0x7f8388069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.455+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f838c085ad0 con 0x7f839c0cccd0 2024-04-25T00:46:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.455+0000 7f839bfff640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f83880678c0 0x7f8388069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.455+0000 7f839bfff640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f83880678c0 0x7f8388069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f839c15f9b0 tx=0x7f8390008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.459+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f838c0565f0 con 0x7f839c0cccd0 2024-04-25T00:46:03.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f839c075320 con 0x7f839c0cccd0 2024-04-25T00:46:03.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83997fa640 1 -- 172.21.15.129:0/2667102164 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f838c055ef0 con 0x7f839c0cccd0 2024-04-25T00:46:03.621 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:46:03.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f83880678c0 msgr2=0x7f8388069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f83880678c0 0x7f8388069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f839c15f9b0 tx=0x7f8390008040 comp rx=0 tx=0).stop 2024-04-25T00:46:03.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 msgr2=0x7f839c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c15a180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c480 tx=0x7f838c00c940 comp rx=0 tx=0).stop 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 shutdown_connections 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f83880678c0 0x7f8388069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c145960 0x7f839c15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c15c590 0x7f839c0af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 --2- 172.21.15.129:0/2667102164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f839c0cccd0 0x7f839c15a180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 >> 172.21.15.129:0/2667102164 conn(0x7f839c067730 msgr2=0x7f839c0cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 shutdown_connections 2024-04-25T00:46:03.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.619+0000 7f83a2bf8640 1 -- 172.21.15.129:0/2667102164 wait complete. 2024-04-25T00:46:03.641 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:46:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 Processor -- start 2024-04-25T00:46:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 -- start start 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b78147940 0x7f6b781621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 0x7f6b78160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7815fb30 con 0x7f6b7815f6a0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7815fef0 con 0x7f6b78147940 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7fd26640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b781627d0 con 0x7f6b780d05a0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7d2c1640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f6b7815f6a0 0x7f6b780cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46954/0 (socket says 172.21.15.129:46954) 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7d2c1640 1 -- 172.21.15.129:0/2658800159 learned_addr learned my addr 172.21.15.129:0/2658800159 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7dac2640 1 --2- 172.21.15.129:0/2658800159 >> v2:172.21.15.196:3300/0 conn(0x7f6b78147940 0x7f6b781621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 0x7f6b78160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088872437 0 0) 0x7f6b7815fb30 con 0x7f6b7815f6a0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b64003530 con 0x7f6b7815f6a0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1458456083 0 0) 0x7f6b64003530 con 0x7f6b7815f6a0 2024-04-25T00:46:03.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 msgr2=0x7f6b78160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 0x7f6b78160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 >> v2:172.21.15.196:3300/0 conn(0x7f6b78147940 msgr2=0x7f6b781621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 --2- 172.21.15.129:0/2658800159 >> v2:172.21.15.196:3300/0 conn(0x7f6b78147940 0x7f6b781621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b78162ab0 con 0x7f6b7815f6a0 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f6b68001680 con 0x7f6b7815f6a0 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 >> v1:172.21.15.129:6789/0 conn(0x7f6b7815f6a0 legacy=0x7f6b780cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b640039c0 0x7f6b64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 0x7f6b64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7815fef0 con 0x7f6b64004d50 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b781627d0 con 0x7f6b640039c0 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6400b9d0 con 0x7f6b780d05a0 2024-04-25T00:46:03.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7d2c1640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 0x7f6b64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.763+0000 7f6b7dac2640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b640039c0 0x7f6b64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 -- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 msgr2=0x7f6b64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b780d05a0 0x7f6b64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 -- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b640039c0 msgr2=0x7f6b64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b640039c0 0x7f6b64003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 -- 172.21.15.129:0/2658800159 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b64003530 con 0x7f6b64004d50 2024-04-25T00:46:03.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7dac2640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b640039c0 0x7f6b64003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009fc0 tx=0x7f6b7401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73669f865f4daeff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b740b2070 con 0x7f6b64004d50 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b740a5c50 con 0x7f6b64004d50 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7cac0640 1 -- 172.21.15.129:0/2658800159 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74009c10 con 0x7f6b64004d50 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7fd26640 1 -- 172.21.15.129:0/2658800159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64004d50 msgr2=0x7f6b640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/2658800159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009fc0 tx=0x7f6b7401cc40 comp rx=0 tx=0).stop 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7e2c3640 1 -- 172.21.15.129:0/2658800159 reap_dead start 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7fd26640 1 -- 172.21.15.129:0/2658800159 shutdown_connections 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7fd26640 1 -- 172.21.15.129:0/2658800159 >> 172.21.15.129:0/2658800159 conn(0x7f6b780d0f80 msgr2=0x7f6b780d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7fd26640 1 -- 172.21.15.129:0/2658800159 shutdown_connections 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.767+0000 7f6b7fd26640 1 -- 172.21.15.129:0/2658800159 wait complete. 2024-04-25T00:46:03.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 Processor -- start 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- start start 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b64004d50 0x7f6b781ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 0x7f6b780ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b78140530 0x7f6b781409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78160380 con 0x7f6b780aae90 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78163070 con 0x7f6b64004d50 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b78160500 con 0x7f6b78140530 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 0x7f6b780ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 0x7f6b780ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52286/0 (socket says 172.21.15.129:52286) 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 -- 172.21.15.129:0/3803149057 learned_addr learned my addr 172.21.15.129:0/3803149057 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b78140530 0x7f6b781409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7dac2640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b64004d50 0x7f6b781ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 -- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b78140530 msgr2=0x7f6b781409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b78140530 0x7f6b781409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 -- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b64004d50 msgr2=0x7f6b781ea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b64004d50 0x7f6b781ea8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 -- 172.21.15.129:0/3803149057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7815e0a0 con 0x7f6b780aae90 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7e2c3640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b78140530 0x7f6b781409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7d2c1640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 0x7f6b780ab2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e7b0 tx=0x7f6b6800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800c8d0 con 0x7f6b780aae90 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b68018070 con 0x7f6b780aae90 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b780ab9e0 con 0x7f6b780aae90 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800ccb0 con 0x7f6b780aae90 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b780abfa0 con 0x7f6b780aae90 2024-04-25T00:46:03.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b78062e50 con 0x7f6b780aae90 2024-04-25T00:46:03.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6b6801d460 con 0x7f6b780aae90 2024-04-25T00:46:03.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b6e7fc640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6b5c0678f0 0x7f6b5c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:03.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f6b68085cb0 con 0x7f6b780aae90 2024-04-25T00:46:03.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.771+0000 7f6b7dac2640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6b5c0678f0 0x7f6b5c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:03.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.775+0000 7f6b7dac2640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6b5c0678f0 0x7f6b5c069db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009b40 tx=0x7f6b70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:03.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.775+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6b68056740 con 0x7f6b780aae90 2024-04-25T00:46:03.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.947+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b7815f450 con 0x7f6b780aae90 2024-04-25T00:46:03.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.947+0000 7f6b6e7fc640 1 -- 172.21.15.129:0/3803149057 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f6b68056040 con 0x7f6b780aae90 2024-04-25T00:46:03.950 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:46:03.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6b5c0678f0 msgr2=0x7f6b5c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6b5c0678f0 0x7f6b5c069db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009b40 tx=0x7f6b70008040 comp rx=0 tx=0).stop 2024-04-25T00:46:03.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 msgr2=0x7f6b780ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:03.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 0x7f6b780ab2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e7b0 tx=0x7f6b6800ec70 comp rx=0 tx=0).stop 2024-04-25T00:46:03.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 shutdown_connections 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f6b5c0678f0 0x7f6b5c069db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b78140530 0x7f6b781409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b780aae90 0x7f6b780ab2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 --2- 172.21.15.129:0/3803149057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b64004d50 0x7f6b781ea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 >> 172.21.15.129:0/3803149057 conn(0x7f6b780d0f80 msgr2=0x7f6b78161150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 shutdown_connections 2024-04-25T00:46:03.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:03.951+0000 7f6b7fd26640 1 -- 172.21.15.129:0/3803149057 wait complete. 2024-04-25T00:46:03.968 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-04-25T00:46:03.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:03.968 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T00:46:03.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33541]: fuse finished with error 0 and tester_r 0 2024-04-25T00:46:03.992 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:04.036 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:46:04.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:04.036 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T00:46:04.088 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:46:04.088 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:46:04.135 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:46:04.136 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:46:04.183 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:46:04.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.303+0000 7f3b50415640 1 Processor -- start 2024-04-25T00:46:04.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.303+0000 7f3b50415640 1 -- start start 2024-04-25T00:46:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3b480aa0c0 0x7f3b480ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3b480aa550 0x7f3b480cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4806d1b0 con 0x7f3b4815bf50 2024-04-25T00:46:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4815ca40 con 0x7f3b480aa0c0 2024-04-25T00:46:04.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4806d490 con 0x7f3b480aa550 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d9b0640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3b480aa0c0 0x7f3b480ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4e1b1640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f3b4815bf50 0x7f3b4815c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46968/0 (socket says 172.21.15.129:46968) 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4e1b1640 1 -- 172.21.15.129:0/1256239476 learned_addr learned my addr 172.21.15.129:0/1256239476 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d9b0640 1 -- 172.21.15.129:0/1256239476 >> v2:172.21.15.196:3300/0 conn(0x7f3b480aa550 msgr2=0x7f3b480cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4e9b2640 1 --2- 172.21.15.129:0/1256239476 >> v2:172.21.15.196:3300/0 conn(0x7f3b480aa550 0x7f3b480cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d1af640 1 -- 172.21.15.129:0/1256239476 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931066155 0 0) 0x7f3b4806d1b0 con 0x7f3b4815bf50 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d9b0640 1 --2- 172.21.15.129:0/1256239476 >> v2:172.21.15.196:3300/0 conn(0x7f3b480aa550 0x7f3b480cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d9b0640 1 -- 172.21.15.129:0/1256239476 >> v1:172.21.15.129:6789/0 conn(0x7f3b4815bf50 legacy=0x7f3b4815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d9b0640 1 -- 172.21.15.129:0/1256239476 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4806d770 con 0x7f3b480aa0c0 2024-04-25T00:46:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d9b0640 1 --2- 172.21.15.129:0/1256239476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b480ccb50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b480b2170 tx=0x7f3b4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cea649eed3f2455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d1af640 1 -- 172.21.15.129:0/1256239476 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440a7070 con 0x7f3b480aa0c0 2024-04-25T00:46:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d1af640 1 -- 172.21.15.129:0/1256239476 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b440b0070 con 0x7f3b480aa0c0 2024-04-25T00:46:04.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b4d1af640 1 -- 172.21.15.129:0/1256239476 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440ab890 con 0x7f3b480aa0c0 2024-04-25T00:46:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- 172.21.15.129:0/1256239476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 msgr2=0x7f3b480ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 --2- 172.21.15.129:0/1256239476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b480ccb50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b480b2170 tx=0x7f3b4401c450 comp rx=0 tx=0).stop 2024-04-25T00:46:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- 172.21.15.129:0/1256239476 shutdown_connections 2024-04-25T00:46:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 --2- 172.21.15.129:0/1256239476 >> v2:172.21.15.196:3300/0 conn(0x7f3b480aa550 0x7f3b480cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 --2- 172.21.15.129:0/1256239476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b480ccb50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- 172.21.15.129:0/1256239476 >> 172.21.15.129:0/1256239476 conn(0x7f3b480b14a0 msgr2=0x7f3b480b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- 172.21.15.129:0/1256239476 shutdown_connections 2024-04-25T00:46:04.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.307+0000 7f3b50415640 1 -- 172.21.15.129:0/1256239476 wait complete. 2024-04-25T00:46:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 Processor -- start 2024-04-25T00:46:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- start start 2024-04-25T00:46:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b48146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480aa550 0x7f3b48149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815bf50 0x7f3b48147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806ebe0 con 0x7f3b480aa0c0 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806ea60 con 0x7f3b4815bf50 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4806ed60 con 0x7f3b480aa550 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b48146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b48146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52308/0 (socket says 172.21.15.129:52308) 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 -- 172.21.15.129:0/3065018375 learned_addr learned my addr 172.21.15.129:0/3065018375 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 -- 172.21.15.129:0/3065018375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480aa550 msgr2=0x7f3b48149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480aa550 0x7f3b48149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 -- 172.21.15.129:0/3065018375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815bf50 msgr2=0x7f3b48147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815bf50 0x7f3b48147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 -- 172.21.15.129:0/3065018375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481498d0 con 0x7f3b480aa0c0 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b4e1b1640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b48146eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000c7c0 tx=0x7f3b4000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40018070 con 0x7f3b480aa0c0 2024-04-25T00:46:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4000a5e0 con 0x7f3b480aa0c0 2024-04-25T00:46:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4814a110 con 0x7f3b480aa0c0 2024-04-25T00:46:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4000a9d0 con 0x7f3b480aa0c0 2024-04-25T00:46:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481575a0 con 0x7f3b480aa0c0 2024-04-25T00:46:04.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.311+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b48062e50 con 0x7f3b480aa0c0 2024-04-25T00:46:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.315+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b4001b070 con 0x7f3b480aa0c0 2024-04-25T00:46:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.315+0000 7f3b3affd640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3b2c0678c0 0x7f3b2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.315+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f3b40085300 con 0x7f3b480aa0c0 2024-04-25T00:46:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.315+0000 7f3b4d9b0640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3b2c0678c0 0x7f3b2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.315+0000 7f3b4d9b0640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3b2c0678c0 0x7f3b2c069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b440af000 tx=0x7f3b44004740 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.319+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3b40055d90 con 0x7f3b480aa0c0 2024-04-25T00:46:04.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.467+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b48075320 con 0x7f3b480aa0c0 2024-04-25T00:46:04.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.467+0000 7f3b3affd640 1 -- 172.21.15.129:0/3065018375 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f3b40055690 con 0x7f3b480aa0c0 2024-04-25T00:46:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:04.471 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3b2c0678c0 msgr2=0x7f3b2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3b2c0678c0 0x7f3b2c069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b440af000 tx=0x7f3b44004740 comp rx=0 tx=0).stop 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 msgr2=0x7f3b48146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b48146eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000c7c0 tx=0x7f3b4000cc80 comp rx=0 tx=0).stop 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 shutdown_connections 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3b2c0678c0 0x7f3b2c069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480aa550 0x7f3b48149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815bf50 0x7f3b48147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 --2- 172.21.15.129:0/3065018375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b480aa0c0 0x7f3b48146eb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 >> 172.21.15.129:0/3065018375 conn(0x7f3b480b14a0 msgr2=0x7f3b4814bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 shutdown_connections 2024-04-25T00:46:04.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.471+0000 7f3b50415640 1 -- 172.21.15.129:0/3065018375 wait complete. 2024-04-25T00:46:04.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:46:04.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 Processor -- start 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 -- start start 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff6b415bd90 0x7ff6b40cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415c170 0x7ff6b40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff6b406f370 0x7ff6b406f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b40cc930 con 0x7ff6b406f370 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b40cd4b0 con 0x7ff6b415bd90 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6bc030640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b406fd60 con 0x7ff6b415c170 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff6b406f370 0x7ff6b406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff6b406f370 0x7ff6b406f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52318/0 (socket says 172.21.15.129:52318) 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/4014792062 learned_addr learned my addr 172.21.15.129:0/4014792062 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6b9dcc640 1 --2- 172.21.15.129:0/4014792062 >> v2:172.21.15.196:3300/0 conn(0x7ff6b415bd90 0x7ff6b40cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6b95cb640 1 --2- 172.21.15.129:0/4014792062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415c170 0x7ff6b40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/4014792062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415c170 msgr2=0x7ff6b40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 --2- 172.21.15.129:0/4014792062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415c170 0x7ff6b40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/4014792062 >> v2:172.21.15.196:3300/0 conn(0x7ff6b415bd90 msgr2=0x7ff6b40cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 --2- 172.21.15.129:0/4014792062 >> v2:172.21.15.196:3300/0 conn(0x7ff6b415bd90 0x7ff6b40cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.595+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/4014792062 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b40ccb80 con 0x7ff6b406f370 2024-04-25T00:46:04.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6ba5cd640 1 --2- 172.21.15.129:0/4014792062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b406f370 0x7ff6b406f820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40cdcc0 tx=0x7ff6ac01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7929bd59e1f78e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6b8dca640 1 -- 172.21.15.129:0/4014792062 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0a56d0 con 0x7ff6b406f370 2024-04-25T00:46:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6b8dca640 1 -- 172.21.15.129:0/4014792062 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac0a5870 con 0x7ff6b406f370 2024-04-25T00:46:04.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6b8dca640 1 -- 172.21.15.129:0/4014792062 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0a5c40 con 0x7ff6b406f370 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 -- 172.21.15.129:0/4014792062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b406f370 msgr2=0x7ff6b406f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 --2- 172.21.15.129:0/4014792062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b406f370 0x7ff6b406f820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40cdcc0 tx=0x7ff6ac01c960 comp rx=0 tx=0).stop 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 -- 172.21.15.129:0/4014792062 shutdown_connections 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 --2- 172.21.15.129:0/4014792062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415c170 0x7ff6b40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 --2- 172.21.15.129:0/4014792062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b406f370 0x7ff6b406f820 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 --2- 172.21.15.129:0/4014792062 >> v2:172.21.15.196:3300/0 conn(0x7ff6b415bd90 0x7ff6b40cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 -- 172.21.15.129:0/4014792062 >> 172.21.15.129:0/4014792062 conn(0x7ff6b4067730 msgr2=0x7ff6b40b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 -- 172.21.15.129:0/4014792062 shutdown_connections 2024-04-25T00:46:04.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.599+0000 7ff6bc030640 1 -- 172.21.15.129:0/4014792062 wait complete. 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 Processor -- start 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- start start 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b406f370 0x7ff6b4155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415bd90 0x7ff6b414b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 0x7ff6b415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b415f460 con 0x7ff6b415c170 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b415f2e0 con 0x7ff6b406f370 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b415f5e0 con 0x7ff6b415bd90 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 0x7ff6b415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 0x7ff6b415cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52328/0 (socket says 172.21.15.129:52328) 2024-04-25T00:46:04.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/2715137800 learned_addr learned my addr 172.21.15.129:0/2715137800 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6b95cb640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415bd90 0x7ff6b414b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6b9dcc640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b406f370 0x7ff6b4155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415bd90 msgr2=0x7ff6b414b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415bd90 0x7ff6b414b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b406f370 msgr2=0x7ff6b4155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b406f370 0x7ff6b4155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 -- 172.21.15.129:0/2715137800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b414d800 con 0x7ff6b415c170 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6ba5cd640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 0x7ff6b415cb30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac01b010 tx=0x7ff6ac019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0bc020 con 0x7ff6b415c170 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac0b8670 con 0x7ff6b415c170 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b414dae0 con 0x7ff6b415c170 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0b8a40 con 0x7ff6b415c170 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b414e0a0 con 0x7ff6b415c170 2024-04-25T00:46:04.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b4062e50 con 0x7ff6b415c170 2024-04-25T00:46:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff6ac0bf020 con 0x7ff6b415c170 2024-04-25T00:46:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6aa7fc640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff6980678c0 0x7ff698069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7ff6ac120d80 con 0x7ff6b415c170 2024-04-25T00:46:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.603+0000 7ff6b9dcc640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff6980678c0 0x7ff698069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.607+0000 7ff6b9dcc640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff6980678c0 0x7ff698069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009710 tx=0x7ff6b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.607+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff6ac0c6030 con 0x7ff6b415c170 2024-04-25T00:46:04.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.767+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6b4070840 con 0x7ff6b415c170 2024-04-25T00:46:04.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6aa7fc640 1 -- 172.21.15.129:0/2715137800 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7ff6ac0165e0 con 0x7ff6b415c170 2024-04-25T00:46:04.771 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:46:04.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff6980678c0 msgr2=0x7ff698069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff6980678c0 0x7ff698069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009710 tx=0x7ff6b0008040 comp rx=0 tx=0).stop 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 msgr2=0x7ff6b415cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 0x7ff6b415cb30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac01b010 tx=0x7ff6ac019700 comp rx=0 tx=0).stop 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 shutdown_connections 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7ff6980678c0 0x7ff698069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6b415bd90 0x7ff6b414b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6b415c170 0x7ff6b415cb30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 --2- 172.21.15.129:0/2715137800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6b406f370 0x7ff6b4155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 >> 172.21.15.129:0/2715137800 conn(0x7ff6b4067730 msgr2=0x7ff6b40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 shutdown_connections 2024-04-25T00:46:04.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.771+0000 7ff6bc030640 1 -- 172.21.15.129:0/2715137800 wait complete. 2024-04-25T00:46:04.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:46:04.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.903+0000 7f7654a6c640 1 Processor -- start 2024-04-25T00:46:04.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.903+0000 7f7654a6c640 1 -- start start 2024-04-25T00:46:04.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f7654a6c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 0x7f76500cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f7654a6c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765006dbf0 con 0x7f7650145960 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f7654a6c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765006cfe0 con 0x7f76500cccd0 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f7654a6c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f765006d2c0 con 0x7f765015c590 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764e59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f7650145960 0x7f7650160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46970/0 (socket says 172.21.15.129:46970) 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764e59c640 1 -- 172.21.15.129:0/721043919 learned_addr learned my addr 172.21.15.129:0/721043919 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 857831598 0 0) 0x7f765006dbf0 con 0x7f7650145960 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f763c003530 con 0x7f7650145960 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764dd9b640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 0x7f76500cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3793924707 0 0) 0x7f763c003530 con 0x7f7650145960 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 msgr2=0x7f76500cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 0x7f76500cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 >> v1:172.21.15.196:6789/0 conn(0x7f76500cccd0 legacy=0x7f765006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765006d5a0 con 0x7f7650145960 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f7640001680 con 0x7f7650145960 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 >> v1:172.21.15.129:6789/0 conn(0x7f7650145960 legacy=0x7f7650160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 0x7f763c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f763c004d50 0x7f763c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f765006d2c0 con 0x7f763c004d50 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f763c00bbb0 con 0x7f763c0039f0 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f763c00bdb0 con 0x7f765015c590 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f763c004d50 0x7f763c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764dd9b640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 0x7f763c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764e59c640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 -- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 msgr2=0x7f763c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:04.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765015c590 0x7f763c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 -- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f763c0039f0 msgr2=0x7f763c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:04.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 -- 172.21.15.129:0/721043919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c003530 con 0x7f763c004d50 2024-04-25T00:46:04.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764e59c640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:04.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.907+0000 7f764ed9d640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f763c004d50 0x7f763c0051d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7648001e70 tx=0x7f764801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04cbda9fb40374b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648001a90 con 0x7f763c004d50 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7648003b80 con 0x7f763c004d50 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764d59a640 1 -- 172.21.15.129:0/721043919 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648005080 con 0x7f763c004d50 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/721043919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f763c004d50 msgr2=0x7f763c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 --2- 172.21.15.129:0/721043919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f763c004d50 0x7f763c0051d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7648001e70 tx=0x7f764801c780 comp rx=0 tx=0).stop 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764ed9d640 1 -- 172.21.15.129:0/721043919 reap_dead start 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/721043919 shutdown_connections 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/721043919 >> 172.21.15.129:0/721043919 conn(0x7f7650067730 msgr2=0x7f76500b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/721043919 shutdown_connections 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/721043919 wait complete. 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 Processor -- start 2024-04-25T00:46:04.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- start start 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 0x7f76501e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76501e16b0 0x7f76501f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765006e550 0x7f76501f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f765006cc40 con 0x7f765006e170 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76500cf890 con 0x7f76501e16b0 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f765006cdc0 con 0x7f765006e550 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 0x7f76501e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 0x7f76501e37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52340/0 (socket says 172.21.15.129:52340) 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 -- 172.21.15.129:0/3051801687 learned_addr learned my addr 172.21.15.129:0/3051801687 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764dd9b640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76501e16b0 0x7f76501f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764ed9d640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765006e550 0x7f76501f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 -- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765006e550 msgr2=0x7f76501f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765006e550 0x7f76501f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 -- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76501e16b0 msgr2=0x7f76501f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:04.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76501e16b0 0x7f76501f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 -- 172.21.15.129:0/3051801687 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76501f2190 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764dd9b640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76501e16b0 0x7f76501f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f764e59c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 0x7f76501e37f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7640001000 tx=0x7f764000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764001b070 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7640011040 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76501e1c20 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400143a0 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.911+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76501e21e0 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.915+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7650062e50 con 0x7f765006e170 2024-04-25T00:46:04.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.915+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f764001e070 con 0x7f765006e170 2024-04-25T00:46:04.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.915+0000 7f7646ffd640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f76300678c0 0x7f7630069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:04.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.915+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f7640088900 con 0x7f765006e170 2024-04-25T00:46:04.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.915+0000 7f764dd9b640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f76300678c0 0x7f7630069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:04.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.915+0000 7f764dd9b640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f76300678c0 0x7f7630069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f765004ff30 tx=0x7f7638008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:04.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:04.919+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7640059420 con 0x7f765006e170 2024-04-25T00:46:05.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76500cef60 con 0x7f765006e170 2024-04-25T00:46:05.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7646ffd640 1 -- 172.21.15.129:0/3051801687 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f7640058d20 con 0x7f765006e170 2024-04-25T00:46:05.068 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f76300678c0 msgr2=0x7f7630069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f76300678c0 0x7f7630069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f765004ff30 tx=0x7f7638008040 comp rx=0 tx=0).stop 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 msgr2=0x7f76501e37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 0x7f76501e37f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7640001000 tx=0x7f764000e4d0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 shutdown_connections 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f76300678c0 0x7f7630069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765006e550 0x7f76501f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76501e16b0 0x7f76501f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 --2- 172.21.15.129:0/3051801687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f765006e170 0x7f76501e37f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 >> 172.21.15.129:0/3051801687 conn(0x7f7650067730 msgr2=0x7f76501433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 shutdown_connections 2024-04-25T00:46:05.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.067+0000 7f7654a6c640 1 -- 172.21.15.129:0/3051801687 wait complete. 2024-04-25T00:46:05.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:46:05.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 Processor -- start 2024-04-25T00:46:05.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 -- start start 2024-04-25T00:46:05.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f263c0c46a0 0x7f263c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f263c0c54f0 0x7f263c070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c15c7c0 con 0x7f263c0c3b70 2024-04-25T00:46:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f263c15d0f0 con 0x7f263c0c46a0 2024-04-25T00:46:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f2642107640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f263c0c5900 con 0x7f263c0c54f0 2024-04-25T00:46:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f263c0c46a0 0x7f263c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f263c0c46a0 0x7f263c0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52352/0 (socket says 172.21.15.129:52352) 2024-04-25T00:46:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 -- 172.21.15.129:0/3187450592 learned_addr learned my addr 172.21.15.129:0/3187450592 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263bfff640 1 --2- 172.21.15.129:0/3187450592 >> v2:172.21.15.196:3300/0 conn(0x7f263c0c54f0 0x7f263c070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 -- 172.21.15.129:0/3187450592 >> v2:172.21.15.196:3300/0 conn(0x7f263c0c54f0 msgr2=0x7f263c070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 --2- 172.21.15.129:0/3187450592 >> v2:172.21.15.196:3300/0 conn(0x7f263c0c54f0 0x7f263c070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 -- 172.21.15.129:0/3187450592 >> v1:172.21.15.196:6789/0 conn(0x7f263c0c3b70 legacy=0x7f263c0c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263b7fe640 1 -- 172.21.15.129:0/3187450592 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c15caa0 con 0x7f263c0c46a0 2024-04-25T00:46:05.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.195+0000 7f263bfff640 1 --2- 172.21.15.129:0/3187450592 >> v2:172.21.15.196:3300/0 conn(0x7f263c0c54f0 0x7f263c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:46:05.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263b7fe640 1 --2- 172.21.15.129:0/3187450592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c46a0 0x7f263c0706d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f263c06df80 tx=0x7f263001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c053249ba3c77d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263a7fc640 1 -- 172.21.15.129:0/3187450592 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300aa610 con 0x7f263c0c46a0 2024-04-25T00:46:05.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263a7fc640 1 -- 172.21.15.129:0/3187450592 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26300af070 con 0x7f263c0c46a0 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263a7fc640 1 -- 172.21.15.129:0/3187450592 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300aa9d0 con 0x7f263c0c46a0 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- 172.21.15.129:0/3187450592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c46a0 msgr2=0x7f263c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 --2- 172.21.15.129:0/3187450592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c46a0 0x7f263c0706d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f263c06df80 tx=0x7f263001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- 172.21.15.129:0/3187450592 shutdown_connections 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 --2- 172.21.15.129:0/3187450592 >> v2:172.21.15.196:3300/0 conn(0x7f263c0c54f0 0x7f263c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 --2- 172.21.15.129:0/3187450592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c46a0 0x7f263c0706d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- 172.21.15.129:0/3187450592 >> 172.21.15.129:0/3187450592 conn(0x7f263c0d12c0 msgr2=0x7f263c0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- 172.21.15.129:0/3187450592 shutdown_connections 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- 172.21.15.129:0/3187450592 wait complete. 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 Processor -- start 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- start start 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 0x7f263c14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0c46a0 0x7f263c0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c0c54f0 0x7f263c14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f263c1603f0 con 0x7f263c0c3b70 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f263c160270 con 0x7f263c0c54f0 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f2642107640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f263c160570 con 0x7f263c0c46a0 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 0x7f263c14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 0x7f263c14bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52364/0 (socket says 172.21.15.129:52364) 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.199+0000 7f263b7fe640 1 -- 172.21.15.129:0/2783385274 learned_addr learned my addr 172.21.15.129:0/2783385274 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:05.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263b7fe640 1 -- 172.21.15.129:0/2783385274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0c46a0 msgr2=0x7f263c0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263b7fe640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0c46a0 0x7f263c0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263b7fe640 1 -- 172.21.15.129:0/2783385274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c0c54f0 msgr2=0x7f263c14c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263b7fe640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c0c54f0 0x7f263c14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263b7fe640 1 -- 172.21.15.129:0/2783385274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c158e30 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263b7fe640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 0x7f263c14bef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f26300ac040 tx=0x7f263001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300aae60 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26300bd610 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c1590d0 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300bda00 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c159970 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c1594c0 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f26300b33f0 con 0x7f263c0c3b70 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2623fff640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f26280678c0 0x7f2628069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f2630120150 con 0x7f263c0c3b70 2024-04-25T00:46:05.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263affd640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f26280678c0 0x7f2628069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.203+0000 7f263affd640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f26280678c0 0x7f2628069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f262c000f70 tx=0x7f262c0079e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.207+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f26300c3060 con 0x7f263c0c3b70 2024-04-25T00:46:05.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f263c1592a0 con 0x7f263c0c3b70 2024-04-25T00:46:05.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2623fff640 1 -- 172.21.15.129:0/2783385274 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f26300a7070 con 0x7f263c0c3b70 2024-04-25T00:46:05.356 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f26280678c0 msgr2=0x7f2628069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f26280678c0 0x7f2628069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f262c000f70 tx=0x7f262c0079e0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 msgr2=0x7f263c14bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 0x7f263c14bef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f26300ac040 tx=0x7f263001ad30 comp rx=0 tx=0).stop 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 shutdown_connections 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f26280678c0 0x7f2628069d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0c46a0 0x7f263c0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c0c54f0 0x7f263c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 --2- 172.21.15.129:0/2783385274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f263c0c3b70 0x7f263c14bef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 >> 172.21.15.129:0/2783385274 conn(0x7f263c0d12c0 msgr2=0x7f263c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 shutdown_connections 2024-04-25T00:46:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.355+0000 7f2642107640 1 -- 172.21.15.129:0/2783385274 wait complete. 2024-04-25T00:46:05.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:46:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc52097c640 1 Processor -- start 2024-04-25T00:46:05.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc52097c640 1 -- start start 2024-04-25T00:46:05.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc52097c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc51c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc52097c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51c06d1b0 con 0x7fc51c0cd260 2024-04-25T00:46:05.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc52097c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51c06d490 con 0x7fc51c0aa0c0 2024-04-25T00:46:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc52097c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c06d770 con 0x7fc51c15bf50 2024-04-25T00:46:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51ad9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc51c0cd260 0x7fc51c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46986/0 (socket says 172.21.15.129:46986) 2024-04-25T00:46:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51ad9d640 1 -- 172.21.15.129:0/2357429148 learned_addr learned my addr 172.21.15.129:0/2357429148 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51a59c640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc51c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918108824 0 0) 0x7fc51c06d1b0 con 0x7fc51c0cd260 2024-04-25T00:46:05.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc504003530 con 0x7fc51c0cd260 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 408975539 0 0) 0x7fc504003530 con 0x7fc51c0cd260 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 msgr2=0x7fc51c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc51c15c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 >> v1:172.21.15.196:6789/0 conn(0x7fc51c0aa0c0 legacy=0x7fc51c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c06da50 con 0x7fc51c0cd260 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fc514001a00 con 0x7fc51c0cd260 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 >> v1:172.21.15.129:6789/0 conn(0x7fc51c0cd260 legacy=0x7fc51c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5040039f0 0x7fc504003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc504004810 0x7fc504004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc5040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c06d770 con 0x7fc504004810 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc50400ba80 con 0x7fc5040039f0 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc50400bc80 con 0x7fc51c15bf50 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc519d9b640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc504004810 0x7fc504004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51ad9d640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc5040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51a59c640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5040039f0 0x7fc504003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc519d9b640 1 -- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 msgr2=0x7fc5040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc519d9b640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc5040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc519d9b640 1 -- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5040039f0 msgr2=0x7fc504003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc519d9b640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5040039f0 0x7fc504003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc519d9b640 1 -- 172.21.15.129:0/2357429148 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc504003530 con 0x7fc504004810 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51ad9d640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c15bf50 0x7fc5040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:05.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.495+0000 7fc51a59c640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5040039f0 0x7fc504003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:05.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc519d9b640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc504004810 0x7fc504004c90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc510001750 tx=0x7fc51001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c099e13c6acaeaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5100013a0 con 0x7fc504004810 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5100052f0 con 0x7fc504004810 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc51959a640 1 -- 172.21.15.129:0/2357429148 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510003840 con 0x7fc504004810 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 -- 172.21.15.129:0/2357429148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc504004810 msgr2=0x7fc504004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 --2- 172.21.15.129:0/2357429148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc504004810 0x7fc504004c90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc510001750 tx=0x7fc51001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc51ad9d640 1 -- 172.21.15.129:0/2357429148 reap_dead start 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 -- 172.21.15.129:0/2357429148 shutdown_connections 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 -- 172.21.15.129:0/2357429148 >> 172.21.15.129:0/2357429148 conn(0x7fc51c0b14a0 msgr2=0x7fc51c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 -- 172.21.15.129:0/2357429148 shutdown_connections 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 -- 172.21.15.129:0/2357429148 wait complete. 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 Processor -- start 2024-04-25T00:46:05.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.499+0000 7fc52097c640 1 -- start start 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc504004810 0x7fc51c147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c1e3160 0x7fc51c135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 0x7fc51c140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c06cee0 con 0x7fc51c1403d0 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c06cd60 con 0x7fc504004810 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c0aa520 con 0x7fc51c1e3160 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 0x7fc51c140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 0x7fc51c140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52386/0 (socket says 172.21.15.129:52386) 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 -- 172.21.15.129:0/2959834597 learned_addr learned my addr 172.21.15.129:0/2959834597 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc519d9b640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c1e3160 0x7fc51c135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 -- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c1e3160 msgr2=0x7fc51c135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c1e3160 0x7fc51c135850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51a59c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc504004810 0x7fc51c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 -- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc504004810 msgr2=0x7fc51c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc504004810 0x7fc51c147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 -- 172.21.15.129:0/2959834597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c14a2a0 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc519d9b640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c1e3160 0x7fc51c135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51ad9d640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 0x7fc51c140850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0b2190 tx=0x7fc514010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51401a070 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514011040 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c142530 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51400eca0 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c142af0 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51c142cf0 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc51401d070 con 0x7fc51c1403d0 2024-04-25T00:46:05.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc50affd640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc4fc0678c0 0x7fc4fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fc514088420 con 0x7fc51c1403d0 2024-04-25T00:46:05.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51a59c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc4fc0678c0 0x7fc4fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.503+0000 7fc51a59c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc4fc0678c0 0x7fc4fc069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099e0 tx=0x7fc50c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.507+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc51c142cf0 con 0x7fc51c1403d0 2024-04-25T00:46:05.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc51c062e50 con 0x7fc51c1403d0 2024-04-25T00:46:05.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc50affd640 1 -- 172.21.15.129:0/2959834597 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fc514058eb0 con 0x7fc51c1403d0 2024-04-25T00:46:05.672 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc4fc0678c0 msgr2=0x7fc4fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc4fc0678c0 0x7fc4fc069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0099e0 tx=0x7fc50c008040 comp rx=0 tx=0).stop 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 msgr2=0x7fc51c140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 0x7fc51c140850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0b2190 tx=0x7fc514010d70 comp rx=0 tx=0).stop 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 shutdown_connections 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc4fc0678c0 0x7fc4fc069d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc51c1e3160 0x7fc51c135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc51c1403d0 0x7fc51c140850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 --2- 172.21.15.129:0/2959834597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc504004810 0x7fc51c147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 >> 172.21.15.129:0/2959834597 conn(0x7fc51c0b14a0 msgr2=0x7fc51c14e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 shutdown_connections 2024-04-25T00:46:05.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.671+0000 7fc52097c640 1 -- 172.21.15.129:0/2959834597 wait complete. 2024-04-25T00:46:05.691 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T00:46:05.691 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:46:05.698 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:46:05.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 Processor -- start 2024-04-25T00:46:05.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 -- start start 2024-04-25T00:46:05.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcb040aa0c0 0x7fcb040ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcb040aa550 0x7fcb040cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb0406d1b0 con 0x7fcb0415bf50 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0415ca40 con 0x7fcb040aa550 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb0b531640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0406d490 con 0x7fcb040aa0c0 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.811+0000 7fcb09ace640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcb040aa550 0x7fcb040cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcb040aa550 0x7fcb040cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52396/0 (socket says 172.21.15.129:52396) 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 -- 172.21.15.129:0/3597995048 learned_addr learned my addr 172.21.15.129:0/3597995048 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb08acc640 1 --2- 172.21.15.129:0/3597995048 >> v2:172.21.15.196:3300/0 conn(0x7fcb040aa0c0 0x7fcb040ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 -- 172.21.15.129:0/3597995048 >> v2:172.21.15.196:3300/0 conn(0x7fcb040aa0c0 msgr2=0x7fcb040ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 --2- 172.21.15.129:0/3597995048 >> v2:172.21.15.196:3300/0 conn(0x7fcb040aa0c0 0x7fcb040ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 -- 172.21.15.129:0/3597995048 >> v1:172.21.15.196:6789/0 conn(0x7fcb0415bf50 legacy=0x7fcb0415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:05.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 -- 172.21.15.129:0/3597995048 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0406d770 con 0x7fcb040aa550 2024-04-25T00:46:05.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb09ace640 1 --2- 172.21.15.129:0/3597995048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb040cd090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb040b3ae0 tx=0x7fcafc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6f0ebe4aa81d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcafbfff640 1 -- 172.21.15.129:0/3597995048 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0a5880 con 0x7fcb040aa550 2024-04-25T00:46:05.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcafbfff640 1 -- 172.21.15.129:0/3597995048 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcafc0a5a20 con 0x7fcb040aa550 2024-04-25T00:46:05.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcafbfff640 1 -- 172.21.15.129:0/3597995048 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0a5df0 con 0x7fcb040aa550 2024-04-25T00:46:05.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 -- 172.21.15.129:0/3597995048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 msgr2=0x7fcb040cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 --2- 172.21.15.129:0/3597995048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb040cd090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb040b3ae0 tx=0x7fcafc01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 -- 172.21.15.129:0/3597995048 shutdown_connections 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 --2- 172.21.15.129:0/3597995048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb040cd090 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 --2- 172.21.15.129:0/3597995048 >> v2:172.21.15.196:3300/0 conn(0x7fcb040aa0c0 0x7fcb040ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 -- 172.21.15.129:0/3597995048 >> 172.21.15.129:0/3597995048 conn(0x7fcb040b14a0 msgr2=0x7fcb040b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 -- 172.21.15.129:0/3597995048 shutdown_connections 2024-04-25T00:46:05.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.815+0000 7fcb0b531640 1 -- 172.21.15.129:0/3597995048 wait complete. 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 Processor -- start 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- start start 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb040aa0c0 0x7fcb0414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb04146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb0415bf50 0x7fcb04153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0406ebe0 con 0x7fcb040aa550 2024-04-25T00:46:05.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0406ea60 con 0x7fcb040aa0c0 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcb0406ed60 con 0x7fcb0415bf50 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb04146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb04146c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52406/0 (socket says 172.21.15.129:52406) 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 -- 172.21.15.129:0/906383545 learned_addr learned my addr 172.21.15.129:0/906383545 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb09ace640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb0415bf50 0x7fcb04153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb092cd640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb040aa0c0 0x7fcb0414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 -- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb0415bf50 msgr2=0x7fcb04153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb0415bf50 0x7fcb04153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 -- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb040aa0c0 msgr2=0x7fcb0414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb040aa0c0 0x7fcb0414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 -- 172.21.15.129:0/906383545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb04147160 con 0x7fcb040aa550 2024-04-25T00:46:05.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb08acc640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb04146c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400d800 tx=0x7fcaf400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400b6c0 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf4019070 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb04153d00 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400ba60 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb041542c0 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb04062e50 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcaf400bc00 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcaf9ffb640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcae80678c0 0x7fcae8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fcaf4086210 con 0x7fcb040aa550 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.819+0000 7fcb092cd640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcae80678c0 0x7fcae8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:05.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.823+0000 7fcb092cd640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcae80678c0 0x7fcae8069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb041f8210 tx=0x7fcb00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:05.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.823+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcaf4056ca0 con 0x7fcb040aa550 2024-04-25T00:46:05.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.979+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb040cd3b0 con 0x7fcb040aa550 2024-04-25T00:46:05.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcaf9ffb640 1 -- 172.21.15.129:0/906383545 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7fcaf40565a0 con 0x7fcb040aa550 2024-04-25T00:46:05.984 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:05.984 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcae80678c0 msgr2=0x7fcae8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcae80678c0 0x7fcae8069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb041f8210 tx=0x7fcb00009040 comp rx=0 tx=0).stop 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 msgr2=0x7fcb04146c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb04146c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400d800 tx=0x7fcaf400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 shutdown_connections 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcae80678c0 0x7fcae8069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb0415bf50 0x7fcb04153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb040aa550 0x7fcb04146c20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 --2- 172.21.15.129:0/906383545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb040aa0c0 0x7fcb0414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:05.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 >> 172.21.15.129:0/906383545 conn(0x7fcb040b14a0 msgr2=0x7fcb040cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:05.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 shutdown_connections 2024-04-25T00:46:05.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:05.983+0000 7fcb0b531640 1 -- 172.21.15.129:0/906383545 wait complete. 2024-04-25T00:46:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:42:46.680195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:42:56.613344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:42:56.989839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:46:06.002 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:46:06.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.119+0000 7f0d7593c640 1 Processor -- start 2024-04-25T00:46:06.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.119+0000 7f0d7593c640 1 -- start start 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0d70145960 0x7f0d70160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0d700ccea0 0x7f0d7006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d7006dbf0 con 0x7f0d7015c590 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0d70160ad0 con 0x7f0d700ccea0 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7006cfe0 con 0x7f0d70145960 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6e7fc640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0d7015c590 0x7f0d700cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:46992/0 (socket says 172.21.15.129:46992) 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6f7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0d700ccea0 0x7f0d7006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6e7fc640 1 -- 172.21.15.129:0/3958979570 learned_addr learned my addr 172.21.15.129:0/3958979570 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/3958979570 >> v2:172.21.15.196:3300/0 conn(0x7f0d70145960 msgr2=0x7f0d70160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6f7fe640 1 --2- 172.21.15.129:0/3958979570 >> v2:172.21.15.196:3300/0 conn(0x7f0d70145960 0x7f0d70160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/3958979570 >> v1:172.21.15.129:6789/0 conn(0x7f0d7015c590 legacy=0x7f0d700cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/3958979570 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7006d2c0 con 0x7f0d700ccea0 2024-04-25T00:46:06.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6f7fe640 1 --2- 172.21.15.129:0/3958979570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d700ccea0 0x7f0d7006e080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0d70073670 tx=0x7f0d6801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd5335058e4bfad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6dffb640 1 -- 172.21.15.129:0/3958979570 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680b2070 con 0x7f0d700ccea0 2024-04-25T00:46:06.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6dffb640 1 -- 172.21.15.129:0/3958979570 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d680a5930 con 0x7f0d700ccea0 2024-04-25T00:46:06.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d6dffb640 1 -- 172.21.15.129:0/3958979570 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680a5d00 con 0x7f0d700ccea0 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- 172.21.15.129:0/3958979570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d700ccea0 msgr2=0x7f0d7006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 --2- 172.21.15.129:0/3958979570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d700ccea0 0x7f0d7006e080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0d70073670 tx=0x7f0d6801c870 comp rx=0 tx=0).stop 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- 172.21.15.129:0/3958979570 shutdown_connections 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 --2- 172.21.15.129:0/3958979570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d700ccea0 0x7f0d7006e080 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 --2- 172.21.15.129:0/3958979570 >> v2:172.21.15.196:3300/0 conn(0x7f0d70145960 0x7f0d70160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- 172.21.15.129:0/3958979570 >> 172.21.15.129:0/3958979570 conn(0x7f0d70067730 msgr2=0x7f0d700b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- 172.21.15.129:0/3958979570 shutdown_connections 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.123+0000 7f0d7593c640 1 -- 172.21.15.129:0/3958979570 wait complete. 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 Processor -- start 2024-04-25T00:46:06.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- start start 2024-04-25T00:46:06.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d700ccea0 0x7f0d7015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d70145960 0x7f0d70166630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 0x7f0d7015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7006cdc0 con 0x7f0d7015c590 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7006cc40 con 0x7f0d70145960 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0d70145d40 con 0x7f0d700ccea0 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 0x7f0d7015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 0x7f0d7015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52428/0 (socket says 172.21.15.129:52428) 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/2762926992 learned_addr learned my addr 172.21.15.129:0/2762926992 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6effd640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d700ccea0 0x7f0d7015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6e7fc640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d70145960 0x7f0d70166630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d700ccea0 msgr2=0x7f0d7015b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d700ccea0 0x7f0d7015b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d70145960 msgr2=0x7f0d70166630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d70145960 0x7f0d70166630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 -- 172.21.15.129:0/2762926992 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d70166b70 con 0x7f0d7015c590 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6effd640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d700ccea0 0x7f0d7015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:06.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d6f7fe640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 0x7f0d7015dc90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d680b0040 tx=0x7f0d68019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680bd020 con 0x7f0d7015c590 2024-04-25T00:46:06.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d680b8670 con 0x7f0d7015c590 2024-04-25T00:46:06.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680b2040 con 0x7f0d7015c590 2024-04-25T00:46:06.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7015cbf0 con 0x7f0d7015c590 2024-04-25T00:46:06.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7015d1b0 con 0x7f0d7015c590 2024-04-25T00:46:06.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d70062e50 con 0x7f0d7015c590 2024-04-25T00:46:06.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.127+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0d680c0020 con 0x7f0d7015c590 2024-04-25T00:46:06.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.131+0000 7f0d67fff640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0d500678f0 0x7f0d50069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.131+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f0d68120770 con 0x7f0d7015c590 2024-04-25T00:46:06.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.131+0000 7f0d6effd640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0d500678f0 0x7f0d50069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.131+0000 7f0d6effd640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0d500678f0 0x7f0d50069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000a8b0 tx=0x7f0d60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.135+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0d680c8020 con 0x7f0d7015c590 2024-04-25T00:46:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.279+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d700b0f00 con 0x7f0d7015c590 2024-04-25T00:46:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d67fff640 1 -- 172.21.15.129:0/2762926992 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f0d68016520 con 0x7f0d7015c590 2024-04-25T00:46:06.284 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0d500678f0 msgr2=0x7f0d50069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0d500678f0 0x7f0d50069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000a8b0 tx=0x7f0d60008040 comp rx=0 tx=0).stop 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 msgr2=0x7f0d7015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 0x7f0d7015dc90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d680b0040 tx=0x7f0d68019610 comp rx=0 tx=0).stop 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 shutdown_connections 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0d500678f0 0x7f0d50069db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d7015c590 0x7f0d7015dc90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d70145960 0x7f0d70166630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 --2- 172.21.15.129:0/2762926992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d700ccea0 0x7f0d7015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 >> 172.21.15.129:0/2762926992 conn(0x7f0d70067730 msgr2=0x7f0d700b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:06.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 shutdown_connections 2024-04-25T00:46:06.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.283+0000 7f0d7593c640 1 -- 172.21.15.129:0/2762926992 wait complete. 2024-04-25T00:46:06.301 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:46:06.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 Processor -- start 2024-04-25T00:46:06.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 -- start start 2024-04-25T00:46:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 0x7f3bdc143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3bdc143e80 0x7f3bdc1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bdc0d2710 con 0x7f3bdc141010 2024-04-25T00:46:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3bdc0d29c0 con 0x7f3bdc143e80 2024-04-25T00:46:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3be0eed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3bdc0d2ca0 con 0x7f3bdc144570 2024-04-25T00:46:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3bda59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f3bdc141010 0x7f3bdc0d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:47002/0 (socket says 172.21.15.129:47002) 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3bda59c640 1 -- 172.21.15.129:0/22958304 learned_addr learned my addr 172.21.15.129:0/22958304 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3bd3fff640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 0x7f3bdc143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3bdad9d640 1 --2- 172.21.15.129:0/22958304 >> v2:172.21.15.196:3300/0 conn(0x7f3bdc143e80 0x7f3bdc1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3641611566 0 0) 0x7f3bdc0d2710 con 0x7f3bdc141010 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.415+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bc8003530 con 0x7f3bdc141010 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2925041195 0 0) 0x7f3bc8003530 con 0x7f3bdc141010 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 msgr2=0x7f3bdc143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 0x7f3bdc143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 >> v2:172.21.15.196:3300/0 conn(0x7f3bdc143e80 msgr2=0x7f3bdc1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 --2- 172.21.15.129:0/22958304 >> v2:172.21.15.196:3300/0 conn(0x7f3bdc143e80 0x7f3bdc1418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc0d2f80 con 0x7f3bdc141010 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f3bcc001680 con 0x7f3bdc141010 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 >> v1:172.21.15.129:6789/0 conn(0x7f3bdc141010 legacy=0x7f3bdc0d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc80039c0 0x7f3bc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 0x7f3bc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc80039c0 0x7f3bc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc8004d50 0x7f3bc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc0d29c0 con 0x7f3bc80039c0 2024-04-25T00:46:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc0d2ca0 con 0x7f3bc8004d50 2024-04-25T00:46:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc800ba00 con 0x7f3bdc144570 2024-04-25T00:46:06.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 -- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 msgr2=0x7f3bc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bdc144570 0x7f3bc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 -- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc8004d50 msgr2=0x7f3bc80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc8004d50 0x7f3bc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 -- 172.21.15.129:0/22958304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc8003530 con 0x7f3bc80039c0 2024-04-25T00:46:06.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bda59c640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc80039c0 0x7f3bc8003de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00c2c0 tx=0x7f3bcc01cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d771087d1e0aec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc007d60 con 0x7f3bc80039c0 2024-04-25T00:46:06.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bcc0a9020 con 0x7f3bc80039c0 2024-04-25T00:46:06.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bd9d9b640 1 -- 172.21.15.129:0/22958304 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc005ae0 con 0x7f3bc80039c0 2024-04-25T00:46:06.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3be0eed640 1 -- 172.21.15.129:0/22958304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc80039c0 msgr2=0x7f3bc8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3be0eed640 1 --2- 172.21.15.129:0/22958304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bc80039c0 0x7f3bc8003de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00c2c0 tx=0x7f3bcc01cd40 comp rx=0 tx=0).stop 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3bdad9d640 1 -- 172.21.15.129:0/22958304 reap_dead start 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3be0eed640 1 -- 172.21.15.129:0/22958304 shutdown_connections 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3be0eed640 1 -- 172.21.15.129:0/22958304 >> 172.21.15.129:0/22958304 conn(0x7f3bdc072be0 msgr2=0x7f3bdc0d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3be0eed640 1 -- 172.21.15.129:0/22958304 shutdown_connections 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.419+0000 7f3be0eed640 1 -- 172.21.15.129:0/22958304 wait complete. 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 Processor -- start 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- start start 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc80039c0 0x7f3bdc13ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 0x7f3bdc13b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc071650 0x7f3bdc071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc142780 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc142600 con 0x7f3bdc071650 2024-04-25T00:46:06.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3bdc142900 con 0x7f3bc80039c0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 0x7f3bdc13b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 0x7f3bdc13b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52456/0 (socket says 172.21.15.129:52456) 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 -- 172.21.15.129:0/386033547 learned_addr learned my addr 172.21.15.129:0/386033547 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bda59c640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc80039c0 0x7f3bdc13ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bdad9d640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc071650 0x7f3bdc071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 -- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc80039c0 msgr2=0x7f3bdc13ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc80039c0 0x7f3bdc13ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 -- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc071650 msgr2=0x7f3bdc071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc071650 0x7f3bdc071b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 -- 172.21.15.129:0/386033547 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc13bde0 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bdad9d640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc071650 0x7f3bdc071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd3fff640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 0x7f3bdc13b000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400e780 tx=0x7f3bc400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4017070 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc400ee90 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bdc13c0c0 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400cb80 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bdc13c680 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.423+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bdc062e50 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.427+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3bc401a070 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.427+0000 7f3bd2ffd640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3bbc0678f0 0x7f3bbc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.427+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f3bc4085f00 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.427+0000 7f3bda59c640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3bbc0678f0 0x7f3bbc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.427+0000 7f3bda59c640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3bbc0678f0 0x7f3bbc069db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0aa000 tx=0x7f3bcc001a50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.431+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3bc4056990 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.587+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3bdc075320 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.587+0000 7f3bd2ffd640 1 -- 172.21.15.129:0/386033547 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f3bc4056290 con 0x7f3bdc13b8e0 2024-04-25T00:46:06.590 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3bbc0678f0 msgr2=0x7f3bbc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3bbc0678f0 0x7f3bbc069db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0aa000 tx=0x7f3bcc001a50 comp rx=0 tx=0).stop 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 msgr2=0x7f3bdc13b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 0x7f3bdc13b000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400e780 tx=0x7f3bc400ec40 comp rx=0 tx=0).stop 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 shutdown_connections 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f3bbc0678f0 0x7f3bbc069db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bdc071650 0x7f3bdc071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bdc13b8e0 0x7f3bdc13b000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 --2- 172.21.15.129:0/386033547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bc80039c0 0x7f3bdc13ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 >> 172.21.15.129:0/386033547 conn(0x7f3bdc072be0 msgr2=0x7f3bdc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:06.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 shutdown_connections 2024-04-25T00:46:06.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.591+0000 7f3be0eed640 1 -- 172.21.15.129:0/386033547 wait complete. 2024-04-25T00:46:06.608 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:46:06.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 Processor -- start 2024-04-25T00:46:06.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 -- start start 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc9c4145960 0x7fc9c4160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 0x7fc9c40cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c406dbf0 con 0x7fc9c40cccd0 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c40cd140 con 0x7fc9c4145960 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9cc23a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c4160ad0 con 0x7fc9c415c590 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9ca7d7640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc9c40cccd0 0x7fc9c40cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:47008/0 (socket says 172.21.15.129:47008) 2024-04-25T00:46:06.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9ca7d7640 1 -- 172.21.15.129:0/891845144 learned_addr learned my addr 172.21.15.129:0/891845144 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:06.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254121058 0 0) 0x7fc9c406dbf0 con 0x7fc9c40cccd0 2024-04-25T00:46:06.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c97d5640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 0x7fc9c40cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9b0003530 con 0x7fc9c40cccd0 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/891845144 >> v2:172.21.15.196:3300/0 conn(0x7fc9c4145960 0x7fc9c4160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1428407375 0 0) 0x7fc9b0003530 con 0x7fc9c40cccd0 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 msgr2=0x7fc9c40cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 0x7fc9c40cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 >> v2:172.21.15.196:3300/0 conn(0x7fc9c4145960 msgr2=0x7fc9c4160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 --2- 172.21.15.129:0/891845144 >> v2:172.21.15.196:3300/0 conn(0x7fc9c4145960 0x7fc9c4160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c406f090 con 0x7fc9c40cccd0 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fc9bc001a00 con 0x7fc9c40cccd0 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 >> v1:172.21.15.129:6789/0 conn(0x7fc9c40cccd0 legacy=0x7fc9c40cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9b00039c0 0x7fc9b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9b0004810 0x7fc9b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 0x7fc9b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c40cd140 con 0x7fc9b00039c0 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4160ad0 con 0x7fc9b0004810 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b000bab0 con 0x7fc9c415c590 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9b00039c0 0x7fc9b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c97d5640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9b0004810 0x7fc9b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9ca7d7640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 0x7fc9b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 -- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 msgr2=0x7fc9b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9c415c590 0x7fc9b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 -- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9b0004810 msgr2=0x7fc9b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9b0004810 0x7fc9b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.727+0000 7fc9c9fd6640 1 -- 172.21.15.129:0/891845144 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b0003530 con 0x7fc9b00039c0 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9b00039c0 0x7fc9b0003de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000b9e0 tx=0x7fc9c001c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5283ce0f4cc89a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00b3070 con 0x7fc9b00039c0 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c00ae410 con 0x7fc9b00039c0 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9c8fd4640 1 -- 172.21.15.129:0/891845144 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000b5e0 con 0x7fc9b00039c0 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 -- 172.21.15.129:0/891845144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9b00039c0 msgr2=0x7fc9b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/891845144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9b00039c0 0x7fc9b0003de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000b9e0 tx=0x7fc9c001c570 comp rx=0 tx=0).stop 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9ca7d7640 1 -- 172.21.15.129:0/891845144 reap_dead start 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 -- 172.21.15.129:0/891845144 shutdown_connections 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 -- 172.21.15.129:0/891845144 >> 172.21.15.129:0/891845144 conn(0x7fc9c4067730 msgr2=0x7fc9c40b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 -- 172.21.15.129:0/891845144 shutdown_connections 2024-04-25T00:46:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 -- 172.21.15.129:0/891845144 wait complete. 2024-04-25T00:46:06.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.731+0000 7fc9cc23a640 1 Processor -- start 2024-04-25T00:46:06.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- start start 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9b00039c0 0x7fc9c415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9c41fac60 0x7fc9c40c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 0x7fc9c40ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c40cf890 con 0x7fc9c40adda0 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c406d880 con 0x7fc9c41fac60 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c406cc40 con 0x7fc9b00039c0 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 0x7fc9c40ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 0x7fc9c40ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52460/0 (socket says 172.21.15.129:52460) 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 -- 172.21.15.129:0/3917316377 learned_addr learned my addr 172.21.15.129:0/3917316377 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9b00039c0 0x7fc9c415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9c97d5640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9c41fac60 0x7fc9c40c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 -- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9b00039c0 msgr2=0x7fc9c415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9b00039c0 0x7fc9c415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 -- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9c41fac60 msgr2=0x7fc9c40c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9c41fac60 0x7fc9c40c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 -- 172.21.15.129:0/3917316377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c40b16f0 con 0x7fc9c40adda0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ca7d7640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 0x7fc9c40ae220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4072920 tx=0x7fc9bc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc007ae0 con 0x7fc9c40adda0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9bc018070 con 0x7fc9c40adda0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c4070830 con 0x7fc9c40adda0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc013420 con 0x7fc9c40adda0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c40c4400 con 0x7fc9c40adda0 2024-04-25T00:46:06.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c4062e50 con 0x7fc9c40adda0 2024-04-25T00:46:06.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9bc0135c0 con 0x7fc9c40adda0 2024-04-25T00:46:06.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ba7fc640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc9a80678c0 0x7fc9a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:06.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fc9bc086100 con 0x7fc9c40adda0 2024-04-25T00:46:06.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.735+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc9a80678c0 0x7fc9a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:06.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.739+0000 7fc9c9fd6640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc9a80678c0 0x7fc9a8069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00b0000 tx=0x7fc9c00046d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:06.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.739+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc9bc056b90 con 0x7fc9c40adda0 2024-04-25T00:46:06.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.887+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc9c4075320 con 0x7fc9c40adda0 2024-04-25T00:46:06.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.887+0000 7fc9ba7fc640 1 -- 172.21.15.129:0/3917316377 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc9bc056490 con 0x7fc9c40adda0 2024-04-25T00:46:06.890 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc9a80678c0 msgr2=0x7fc9a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc9a80678c0 0x7fc9a8069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00b0000 tx=0x7fc9c00046d0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 msgr2=0x7fc9c40ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 0x7fc9c40ae220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4072920 tx=0x7fc9bc00e530 comp rx=0 tx=0).stop 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 shutdown_connections 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fc9a80678c0 0x7fc9a8069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9c40adda0 0x7fc9c40ae220 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9c41fac60 0x7fc9c40c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 --2- 172.21.15.129:0/3917316377 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9b00039c0 0x7fc9c415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:06.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 >> 172.21.15.129:0/3917316377 conn(0x7fc9c4067730 msgr2=0x7fc9c40cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:06.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 shutdown_connections 2024-04-25T00:46:06.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:06.891+0000 7fc9cc23a640 1 -- 172.21.15.129:0/3917316377 wait complete. 2024-04-25T00:46:06.908 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:46:07.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a15f84640 1 Processor -- start 2024-04-25T00:46:07.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a15f84640 1 -- start start 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a15f84640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f8a100aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a15f84640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a1006d1b0 con 0x7f8a100ccb50 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a15f84640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a1006d490 con 0x7f8a1015bf50 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a15f84640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1015ca40 con 0x7f8a100aa0c0 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f8a100ccb50 0x7f8a1006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:47020/0 (socket says 172.21.15.129:47020) 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 -- 172.21.15.129:0/1798404874 learned_addr learned my addr 172.21.15.129:0/1798404874 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0effd640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f8a100aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961539238 0 0) 0x7f8a1006d1b0 con 0x7f8a100ccb50 2024-04-25T00:46:07.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89f8003530 con 0x7f8a100ccb50 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1256592628 0 0) 0x7f89f8003530 con 0x7f8a100ccb50 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 msgr2=0x7f8a100aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f8a100aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 >> v1:172.21.15.196:6789/0 conn(0x7f8a1015bf50 legacy=0x7f8a1015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1006d770 con 0x7f8a100ccb50 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f8a08001a00 con 0x7f8a100ccb50 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 >> v1:172.21.15.129:6789/0 conn(0x7f8a100ccb50 legacy=0x7f8a1006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f80039f0 0x7f89f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f89f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f89f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1015ca40 con 0x7f89f8004d50 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89f800ba80 con 0x7f89f80039f0 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f89f800bc80 con 0x7f8a100aa0c0 2024-04-25T00:46:07.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f89f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0effd640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f89f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0f7fe640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f80039f0 0x7f89f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 -- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 msgr2=0x7f89f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f89f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 -- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f80039f0 msgr2=0x7f89f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f80039f0 0x7f89f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0ffff640 1 -- 172.21.15.129:0/1798404874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f8003530 con 0x7f89f8004d50 2024-04-25T00:46:07.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.019+0000 7f8a0effd640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a100aa0c0 0x7f89f8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:46:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0ffff640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f89f80051d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800a830 tx=0x7f8a0801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcce229bcf4f4813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800a480 con 0x7f89f8004d50 2024-04-25T00:46:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a080ad630 con 0x7f89f8004d50 2024-04-25T00:46:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0e7fc640 1 -- 172.21.15.129:0/1798404874 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080ada60 con 0x7f89f8004d50 2024-04-25T00:46:07.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- 172.21.15.129:0/1798404874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 msgr2=0x7f89f80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 --2- 172.21.15.129:0/1798404874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f89f80051d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800a830 tx=0x7f8a0801d360 comp rx=0 tx=0).stop 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0ffff640 1 -- 172.21.15.129:0/1798404874 reap_dead start 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- 172.21.15.129:0/1798404874 shutdown_connections 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- 172.21.15.129:0/1798404874 >> 172.21.15.129:0/1798404874 conn(0x7f8a100b14a0 msgr2=0x7f8a100b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- 172.21.15.129:0/1798404874 shutdown_connections 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- 172.21.15.129:0/1798404874 wait complete. 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 Processor -- start 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- start start 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f8a10159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a101f8570 0x7f8a100b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a101e3ba0 0x7f8a101435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1006cd60 con 0x7f89f8004d50 2024-04-25T00:46:07.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1006dd30 con 0x7f8a101f8570 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a15f84640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1006cee0 con 0x7f8a101e3ba0 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f8a10159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f8a10159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52472/0 (socket says 172.21.15.129:52472) 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0f7fe640 1 -- 172.21.15.129:0/2011747727 learned_addr learned my addr 172.21.15.129:0/2011747727 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0f7fe640 1 -- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a101e3ba0 msgr2=0x7f8a101435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0f7fe640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a101e3ba0 0x7f8a101435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.023+0000 7f8a0effd640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a101f8570 0x7f8a100b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a0f7fe640 1 -- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a101f8570 msgr2=0x7f8a100b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a0f7fe640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a101f8570 0x7f8a100b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a0f7fe640 1 -- 172.21.15.129:0/2011747727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1015b780 con 0x7f89f8004d50 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a0f7fe640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f8a10159660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e480 tx=0x7f8a0000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000052e0 con 0x7f89f8004d50 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a00018070 con 0x7f89f8004d50 2024-04-25T00:46:07.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a101e58a0 con 0x7f89f8004d50 2024-04-25T00:46:07.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00003820 con 0x7f89f8004d50 2024-04-25T00:46:07.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1015b4a0 con 0x7f89f8004d50 2024-04-25T00:46:07.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a10062e50 con 0x7f89f8004d50 2024-04-25T00:46:07.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8a000039c0 con 0x7f89f8004d50 2024-04-25T00:46:07.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f89f7fff640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f89fc0678f0 0x7f89fc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:07.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a0effd640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f89fc0678f0 0x7f89fc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:07.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f8a000859a0 con 0x7f89f8004d50 2024-04-25T00:46:07.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.027+0000 7f8a0effd640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f89fc0678f0 0x7f89fc069db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a101e54d0 tx=0x7f8a04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:07.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.031+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8a000564c0 con 0x7f89f8004d50 2024-04-25T00:46:07.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.203+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a10075320 con 0x7f89f8004d50 2024-04-25T00:46:07.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f89f7fff640 1 -- 172.21.15.129:0/2011747727 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f8a00005480 con 0x7f89f8004d50 2024-04-25T00:46:07.208 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T00:46:07.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f89fc0678f0 msgr2=0x7f89fc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f89fc0678f0 0x7f89fc069db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a101e54d0 tx=0x7f8a04008040 comp rx=0 tx=0).stop 2024-04-25T00:46:07.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 msgr2=0x7f8a10159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:07.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f8a10159660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e480 tx=0x7f8a0000e940 comp rx=0 tx=0).stop 2024-04-25T00:46:07.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 shutdown_connections 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f89fc0678f0 0x7f89fc069db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8a101e3ba0 0x7f8a101435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a101f8570 0x7f8a100b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 --2- 172.21.15.129:0/2011747727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89f8004d50 0x7f8a10159660 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 >> 172.21.15.129:0/2011747727 conn(0x7f8a100b14a0 msgr2=0x7f8a101e7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 shutdown_connections 2024-04-25T00:46:07.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:07.207+0000 7f8a15f84640 1 -- 172.21.15.129:0/2011747727 wait complete. 2024-04-25T00:46:07.227 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-04-25T00:46:07.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:07.227 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T00:46:07.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33660]: fuse finished with error 0 and tester_r 0 2024-04-25T00:46:07.243 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:13.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:46:13.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.345 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T00:46:13.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.353 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T00:46:13.399 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T00:46:13.400 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T00:46:13.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T00:46:13.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T00:46:13.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.536 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-25T00:46:13.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.713 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T00:46:13.719 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T00:46:13.719 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:46:13.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.720 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T00:46:13.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:46:13.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:46:13.873 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-04-25T00:46:14.021 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T00:46:14.033 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:46:14.033 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:46:14.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 Processor -- start 2024-04-25T00:46:14.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 -- start start 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f307c15bf50 0x7f307c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c0aa0c0 0x7f307c0aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307c06d1b0 con 0x7f307c0ccb50 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f307c06d490 con 0x7f307c15bf50 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3083a32640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f307c06d770 con 0x7f307c0aa0c0 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f30817ce640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f307c15bf50 0x7f307c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f30817ce640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f307c15bf50 0x7f307c15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38110/0 (socket says 172.21.15.129:38110) 2024-04-25T00:46:14.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f30817ce640 1 -- 172.21.15.129:0/2083542206 learned_addr learned my addr 172.21.15.129:0/2083542206 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:14.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.155+0000 7f3080fcd640 1 --2- 172.21.15.129:0/2083542206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c0aa0c0 0x7f307c0aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f30817ce640 1 -- 172.21.15.129:0/2083542206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c0aa0c0 msgr2=0x7f307c0aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f30817ce640 1 --2- 172.21.15.129:0/2083542206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c0aa0c0 0x7f307c0aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f30817ce640 1 -- 172.21.15.129:0/2083542206 >> v1:172.21.15.196:6789/0 conn(0x7f307c0ccb50 legacy=0x7f307c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f30817ce640 1 -- 172.21.15.129:0/2083542206 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c06da50 con 0x7f307c15bf50 2024-04-25T00:46:14.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f30817ce640 1 --2- 172.21.15.129:0/2083542206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c15bf50 0x7f307c15c330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f307801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7713389343305c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3073fff640 1 -- 172.21.15.129:0/2083542206 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780a7070 con 0x7f307c15bf50 2024-04-25T00:46:14.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3073fff640 1 -- 172.21.15.129:0/2083542206 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30780b0070 con 0x7f307c15bf50 2024-04-25T00:46:14.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3073fff640 1 -- 172.21.15.129:0/2083542206 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780ab890 con 0x7f307c15bf50 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 -- 172.21.15.129:0/2083542206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c15bf50 msgr2=0x7f307c15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 --2- 172.21.15.129:0/2083542206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c15bf50 0x7f307c15c330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f307801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 -- 172.21.15.129:0/2083542206 shutdown_connections 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 --2- 172.21.15.129:0/2083542206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c0aa0c0 0x7f307c0aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 --2- 172.21.15.129:0/2083542206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c15bf50 0x7f307c15c330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 -- 172.21.15.129:0/2083542206 >> 172.21.15.129:0/2083542206 conn(0x7f307c0b14a0 msgr2=0x7f307c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:14.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 -- 172.21.15.129:0/2083542206 shutdown_connections 2024-04-25T00:46:14.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.159+0000 7f3083a32640 1 -- 172.21.15.129:0/2083542206 wait complete. 2024-04-25T00:46:14.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 Processor -- start 2024-04-25T00:46:14.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- start start 2024-04-25T00:46:14.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 0x7f307c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c0ccb50 0x7f307c1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c15bf50 0x7f307c147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f307c06cee0 con 0x7f307c0aa0c0 2024-04-25T00:46:14.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f307c06cd60 con 0x7f307c0ccb50 2024-04-25T00:46:14.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f307c169030 con 0x7f307c15bf50 2024-04-25T00:46:14.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 0x7f307c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 0x7f307c14e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38118/0 (socket says 172.21.15.129:38118) 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 -- 172.21.15.129:0/725738580 learned_addr learned my addr 172.21.15.129:0/725738580 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3081fcf640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c15bf50 0x7f307c147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3080fcd640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c0ccb50 0x7f307c1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 -- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c15bf50 msgr2=0x7f307c147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c15bf50 0x7f307c147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 -- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c0ccb50 msgr2=0x7f307c1478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c0ccb50 0x7f307c1478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 -- 172.21.15.129:0/725738580 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c146340 con 0x7f307c0aa0c0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3080fcd640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c0ccb50 0x7f307c1478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f30817ce640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 0x7f307c14e0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f307801aba0 tx=0x7f307801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780be5f0 con 0x7f307c0aa0c0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30780be790 con 0x7f307c0aa0c0 2024-04-25T00:46:14.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c146620 con 0x7f307c0aa0c0 2024-04-25T00:46:14.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780beb80 con 0x7f307c0aa0c0 2024-04-25T00:46:14.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c146be0 con 0x7f307c0aa0c0 2024-04-25T00:46:14.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307c062e50 con 0x7f307c0aa0c0 2024-04-25T00:46:14.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f30780b43f0 con 0x7f307c0aa0c0 2024-04-25T00:46:14.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3071ffb640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f30600678f0 0x7f3060069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3080fcd640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f30600678f0 0x7f3060069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.163+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f3078120ad0 con 0x7f307c0aa0c0 2024-04-25T00:46:14.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.167+0000 7f3080fcd640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f30600678f0 0x7f3060069db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f307c0aa9d0 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.171+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f30780c4060 con 0x7f307c0aa0c0 2024-04-25T00:46:14.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.327+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f307c062e50 con 0x7f30600678f0 2024-04-25T00:46:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.347+0000 7f3071ffb640 1 -- 172.21.15.129:0/725738580 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252414 (secure 0 0 0) 0x7f307c062e50 con 0x7f30600678f0 2024-04-25T00:46:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:46:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f30600678f0 msgr2=0x7f3060069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f30600678f0 0x7f3060069db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f307c0aa9d0 tx=0x7f306c008040 comp rx=0 tx=0).stop 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 msgr2=0x7f307c14e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 0x7f307c14e0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f307801aba0 tx=0x7f307801ad30 comp rx=0 tx=0).stop 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 shutdown_connections 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f30600678f0 0x7f3060069db0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f307c15bf50 0x7f307c147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f307c0ccb50 0x7f307c1478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 --2- 172.21.15.129:0/725738580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f307c0aa0c0 0x7f307c14e0f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 >> 172.21.15.129:0/725738580 conn(0x7f307c0b14a0 msgr2=0x7f307c0a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 shutdown_connections 2024-04-25T00:46:14.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.351+0000 7f3083a32640 1 -- 172.21.15.129:0/725738580 wait complete. 2024-04-25T00:46:14.371 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-04-25T00:46:12.663777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31557876,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":181,"num_read_kb":491,"num_write":3411,"num_write_kb":33997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7316,"ondisk_log_size":7316,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333704,"kb_used_data":63496,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754641016,"statfs":{"total":773094113280,"available":772752400384,"internally_reserved":0,"allocated":65019904,"data_stored":63782200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1561,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003652"},"pg_stats":[{"pgid":"3.3f","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.546368+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:45:51.546368+0000","last_peered":"2024-04-25T00:45:51.546368+0000","last_clean":"2024-04-25T00:45:51.546368+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:45:51.546368+0000","last_undegraded":"2024-04-25T00:45:51.546368+0000","last_fullsized":"2024-04-25T00:45:51.546368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491744+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:45:51.491744+0000","last_peered":"2024-04-25T00:45:51.491744+0000","last_clean":"2024-04-25T00:45:51.491744+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:45:51.491744+0000","last_undegraded":"2024-04-25T00:45:51.491744+0000","last_fullsized":"2024-04-25T00:45:51.491744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515512+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:45:51.515512+0000","last_peered":"2024-04-25T00:45:51.515512+0000","last_clean":"2024-04-25T00:45:51.515512+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:45:51.515512+0000","last_undegraded":"2024-04-25T00:45:51.515512+0000","last_fullsized":"2024-04-25T00:45:51.515512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'40","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527087+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:45:51.527087+0000","last_peered":"2024-04-25T00:45:51.527087+0000","last_clean":"2024-04-25T00:45:51.527087+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:45:51.527087+0000","last_undegraded":"2024-04-25T00:45:51.527087+0000","last_fullsized":"2024-04-25T00:45:51.527087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.539538+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:45:51.539538+0000","last_peered":"2024-04-25T00:45:51.539538+0000","last_clean":"2024-04-25T00:45:51.539538+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:45:51.539538+0000","last_undegraded":"2024-04-25T00:45:51.539538+0000","last_fullsized":"2024-04-25T00:45:51.539538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527080+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:45:51.527080+0000","last_peered":"2024-04-25T00:45:51.527080+0000","last_clean":"2024-04-25T00:45:51.527080+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:45:51.527080+0000","last_undegraded":"2024-04-25T00:45:51.527080+0000","last_fullsized":"2024-04-25T00:45:51.527080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.545654+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:45:51.545654+0000","last_peered":"2024-04-25T00:45:51.545654+0000","last_clean":"2024-04-25T00:45:51.545654+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:45:51.545654+0000","last_undegraded":"2024-04-25T00:45:51.545654+0000","last_fullsized":"2024-04-25T00:45:51.545654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525802+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:45:51.525802+0000","last_peered":"2024-04-25T00:45:51.525802+0000","last_clean":"2024-04-25T00:45:51.525802+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:45:51.525802+0000","last_undegraded":"2024-04-25T00:45:51.525802+0000","last_fullsized":"2024-04-25T00:45:51.525802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'54","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542405+0000","last_change":"2024-04-25T00:43:39.829539+0000","last_active":"2024-04-25T00:45:51.542405+0000","last_peered":"2024-04-25T00:45:51.542405+0000","last_clean":"2024-04-25T00:45:51.542405+0000","last_became_active":"2024-04-25T00:43:39.828586+0000","last_became_peered":"2024-04-25T00:43:39.828586+0000","last_unstale":"2024-04-25T00:45:51.542405+0000","last_undegraded":"2024-04-25T00:45:51.542405+0000","last_fullsized":"2024-04-25T00:45:51.542405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:31.663001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.540595+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:45:51.540595+0000","last_peered":"2024-04-25T00:45:51.540595+0000","last_clean":"2024-04-25T00:45:51.540595+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:45:51.540595+0000","last_undegraded":"2024-04-25T00:45:51.540595+0000","last_fullsized":"2024-04-25T00:45:51.540595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.528273+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:45:51.528273+0000","last_peered":"2024-04-25T00:45:51.528273+0000","last_clean":"2024-04-25T00:45:51.528273+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:45:51.528273+0000","last_undegraded":"2024-04-25T00:45:51.528273+0000","last_fullsized":"2024-04-25T00:45:51.528273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'44","reported_seq":61,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.532319+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:45:51.532319+0000","last_peered":"2024-04-25T00:45:51.532319+0000","last_clean":"2024-04-25T00:45:51.532319+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:45:51.532319+0000","last_undegraded":"2024-04-25T00:45:51.532319+0000","last_fullsized":"2024-04-25T00:45:51.532319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.512541+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:45:51.512541+0000","last_peered":"2024-04-25T00:45:51.512541+0000","last_clean":"2024-04-25T00:45:51.512541+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:45:51.512541+0000","last_undegraded":"2024-04-25T00:45:51.512541+0000","last_fullsized":"2024-04-25T00:45:51.512541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.519079+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:45:51.519079+0000","last_peered":"2024-04-25T00:45:51.519079+0000","last_clean":"2024-04-25T00:45:51.519079+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:45:51.519079+0000","last_undegraded":"2024-04-25T00:45:51.519079+0000","last_fullsized":"2024-04-25T00:45:51.519079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.495151+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:45:51.495151+0000","last_peered":"2024-04-25T00:45:51.495151+0000","last_clean":"2024-04-25T00:45:51.495151+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:45:51.495151+0000","last_undegraded":"2024-04-25T00:45:51.495151+0000","last_fullsized":"2024-04-25T00:45:51.495151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522025+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:45:51.522025+0000","last_peered":"2024-04-25T00:45:51.522025+0000","last_clean":"2024-04-25T00:45:51.522025+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:45:51.522025+0000","last_undegraded":"2024-04-25T00:45:51.522025+0000","last_fullsized":"2024-04-25T00:45:51.522025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549462+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:45:51.549462+0000","last_peered":"2024-04-25T00:45:51.549462+0000","last_clean":"2024-04-25T00:45:51.549462+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:45:51.549462+0000","last_undegraded":"2024-04-25T00:45:51.549462+0000","last_fullsized":"2024-04-25T00:45:51.549462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.531867+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:45:51.531867+0000","last_peered":"2024-04-25T00:45:51.531867+0000","last_clean":"2024-04-25T00:45:51.531867+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:45:51.531867+0000","last_undegraded":"2024-04-25T00:45:51.531867+0000","last_fullsized":"2024-04-25T00:45:51.531867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533095+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:45:51.533095+0000","last_peered":"2024-04-25T00:45:51.533095+0000","last_clean":"2024-04-25T00:45:51.533095+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:45:51.533095+0000","last_undegraded":"2024-04-25T00:45:51.533095+0000","last_fullsized":"2024-04-25T00:45:51.533095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'62","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515861+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:45:51.515861+0000","last_peered":"2024-04-25T00:45:51.515861+0000","last_clean":"2024-04-25T00:45:51.515861+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:45:51.515861+0000","last_undegraded":"2024-04-25T00:45:51.515861+0000","last_fullsized":"2024-04-25T00:45:51.515861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548225+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:45:51.548225+0000","last_peered":"2024-04-25T00:45:51.548225+0000","last_clean":"2024-04-25T00:45:51.548225+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:45:51.548225+0000","last_undegraded":"2024-04-25T00:45:51.548225+0000","last_fullsized":"2024-04-25T00:45:51.548225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549483+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:45:51.549483+0000","last_peered":"2024-04-25T00:45:51.549483+0000","last_clean":"2024-04-25T00:45:51.549483+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:45:51.549483+0000","last_undegraded":"2024-04-25T00:45:51.549483+0000","last_fullsized":"2024-04-25T00:45:51.549483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510069+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:45:51.510069+0000","last_peered":"2024-04-25T00:45:51.510069+0000","last_clean":"2024-04-25T00:45:51.510069+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:45:51.510069+0000","last_undegraded":"2024-04-25T00:45:51.510069+0000","last_fullsized":"2024-04-25T00:45:51.510069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526220+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:45:51.526220+0000","last_peered":"2024-04-25T00:45:51.526220+0000","last_clean":"2024-04-25T00:45:51.526220+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:45:51.526220+0000","last_undegraded":"2024-04-25T00:45:51.526220+0000","last_fullsized":"2024-04-25T00:45:51.526220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.499400+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:45:51.499400+0000","last_peered":"2024-04-25T00:45:51.499400+0000","last_clean":"2024-04-25T00:45:51.499400+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:45:51.499400+0000","last_undegraded":"2024-04-25T00:45:51.499400+0000","last_fullsized":"2024-04-25T00:45:51.499400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'66","reported_seq":83,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548312+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:45:51.548312+0000","last_peered":"2024-04-25T00:45:51.548312+0000","last_clean":"2024-04-25T00:45:51.548312+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:45:51.548312+0000","last_undegraded":"2024-04-25T00:45:51.548312+0000","last_fullsized":"2024-04-25T00:45:51.548312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'37","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500102+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:45:51.500102+0000","last_peered":"2024-04-25T00:45:51.500102+0000","last_clean":"2024-04-25T00:45:51.500102+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:45:51.500102+0000","last_undegraded":"2024-04-25T00:45:51.500102+0000","last_fullsized":"2024-04-25T00:45:51.500102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509134+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:45:51.509134+0000","last_peered":"2024-04-25T00:45:51.509134+0000","last_clean":"2024-04-25T00:45:51.509134+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:45:51.509134+0000","last_undegraded":"2024-04-25T00:45:51.509134+0000","last_fullsized":"2024-04-25T00:45:51.509134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'52","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525040+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:45:51.525040+0000","last_peered":"2024-04-25T00:45:51.525040+0000","last_clean":"2024-04-25T00:45:51.525040+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:45:51.525040+0000","last_undegraded":"2024-04-25T00:45:51.525040+0000","last_fullsized":"2024-04-25T00:45:51.525040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'58","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500399+0000","last_change":"2024-04-25T00:43:39.818085+0000","last_active":"2024-04-25T00:45:51.500399+0000","last_peered":"2024-04-25T00:45:51.500399+0000","last_clean":"2024-04-25T00:45:51.500399+0000","last_became_active":"2024-04-25T00:43:39.817067+0000","last_became_peered":"2024-04-25T00:43:39.817067+0000","last_unstale":"2024-04-25T00:45:51.500399+0000","last_undegraded":"2024-04-25T00:45:51.500399+0000","last_fullsized":"2024-04-25T00:45:51.500399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.520841+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:45:51.520841+0000","last_peered":"2024-04-25T00:45:51.520841+0000","last_clean":"2024-04-25T00:45:51.520841+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:45:51.520841+0000","last_undegraded":"2024-04-25T00:45:51.520841+0000","last_fullsized":"2024-04-25T00:45:51.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'69","reported_seq":86,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515490+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:45:51.515490+0000","last_peered":"2024-04-25T00:45:51.515490+0000","last_clean":"2024-04-25T00:45:51.515490+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:45:51.515490+0000","last_undegraded":"2024-04-25T00:45:51.515490+0000","last_fullsized":"2024-04-25T00:45:51.515490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'44","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514482+0000","last_change":"2024-04-25T00:43:39.825468+0000","last_active":"2024-04-25T00:45:51.514482+0000","last_peered":"2024-04-25T00:45:51.514482+0000","last_clean":"2024-04-25T00:45:51.514482+0000","last_became_active":"2024-04-25T00:43:39.824614+0000","last_became_peered":"2024-04-25T00:43:39.824614+0000","last_unstale":"2024-04-25T00:45:51.514482+0000","last_undegraded":"2024-04-25T00:45:51.514482+0000","last_fullsized":"2024-04-25T00:45:51.514482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1500","reported_seq":1519,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.578349+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:45:35.578349+0000","last_peered":"2024-04-25T00:45:35.578349+0000","last_clean":"2024-04-25T00:45:35.578349+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:45:35.578349+0000","last_undegraded":"2024-04-25T00:45:35.578349+0000","last_fullsized":"2024-04-25T00:45:35.578349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1500,"log_dups_size":0,"ondisk_log_size":1500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1474,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.530748+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:45:51.530748+0000","last_peered":"2024-04-25T00:45:51.530748+0000","last_clean":"2024-04-25T00:45:51.530748+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:45:51.530748+0000","last_undegraded":"2024-04-25T00:45:51.530748+0000","last_fullsized":"2024-04-25T00:45:51.530748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'36","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588170+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:45:35.588170+0000","last_peered":"2024-04-25T00:45:35.588170+0000","last_clean":"2024-04-25T00:45:35.588170+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:45:35.588170+0000","last_undegraded":"2024-04-25T00:45:35.588170+0000","last_fullsized":"2024-04-25T00:45:35.588170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'19","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.962699+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:45:32.962699+0000","last_peered":"2024-04-25T00:45:32.962699+0000","last_clean":"2024-04-25T00:45:32.962699+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:45:32.962699+0000","last_undegraded":"2024-04-25T00:45:32.962699+0000","last_fullsized":"2024-04-25T00:45:32.962699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'65","reported_seq":82,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516989+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:45:51.516989+0000","last_peered":"2024-04-25T00:45:51.516989+0000","last_clean":"2024-04-25T00:45:51.516989+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:45:51.516989+0000","last_undegraded":"2024-04-25T00:45:51.516989+0000","last_fullsized":"2024-04-25T00:45:51.516989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589858+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:45:35.589858+0000","last_peered":"2024-04-25T00:45:35.589858+0000","last_clean":"2024-04-25T00:45:35.589858+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:45:35.589858+0000","last_undegraded":"2024-04-25T00:45:35.589858+0000","last_fullsized":"2024-04-25T00:45:35.589858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.504832+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:45:51.504832+0000","last_peered":"2024-04-25T00:45:51.504832+0000","last_clean":"2024-04-25T00:45:51.504832+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:45:51.504832+0000","last_undegraded":"2024-04-25T00:45:51.504832+0000","last_fullsized":"2024-04-25T00:45:51.504832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.595446+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:45:35.595446+0000","last_peered":"2024-04-25T00:45:35.595446+0000","last_clean":"2024-04-25T00:45:35.595446+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:45:35.595446+0000","last_undegraded":"2024-04-25T00:45:35.595446+0000","last_fullsized":"2024-04-25T00:45:35.595446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'73","reported_seq":90,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549377+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:45:51.549377+0000","last_peered":"2024-04-25T00:45:51.549377+0000","last_clean":"2024-04-25T00:45:51.549377+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:45:51.549377+0000","last_undegraded":"2024-04-25T00:45:51.549377+0000","last_fullsized":"2024-04-25T00:45:51.549377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599552+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:45:35.599552+0000","last_peered":"2024-04-25T00:45:35.599552+0000","last_clean":"2024-04-25T00:45:35.599552+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:45:35.599552+0000","last_undegraded":"2024-04-25T00:45:35.599552+0000","last_fullsized":"2024-04-25T00:45:35.599552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521602+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:45:51.521602+0000","last_peered":"2024-04-25T00:45:51.521602+0000","last_clean":"2024-04-25T00:45:51.521602+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:45:51.521602+0000","last_undegraded":"2024-04-25T00:45:51.521602+0000","last_fullsized":"2024-04-25T00:45:51.521602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'55","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.142036+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:45:50.142036+0000","last_peered":"2024-04-25T00:45:50.142036+0000","last_clean":"2024-04-25T00:45:50.142036+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:45:50.142036+0000","last_undegraded":"2024-04-25T00:45:50.142036+0000","last_fullsized":"2024-04-25T00:45:50.142036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526491+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:45:51.526491+0000","last_peered":"2024-04-25T00:45:51.526491+0000","last_clean":"2024-04-25T00:45:51.526491+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:45:51.526491+0000","last_undegraded":"2024-04-25T00:45:51.526491+0000","last_fullsized":"2024-04-25T00:45:51.526491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'927","reported_seq":946,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:47.871752+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:45:47.871752+0000","last_peered":"2024-04-25T00:45:47.871752+0000","last_clean":"2024-04-25T00:45:47.871752+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:45:47.871752+0000","last_undegraded":"2024-04-25T00:45:47.871752+0000","last_fullsized":"2024-04-25T00:45:47.871752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":908,"num_write_kb":4392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538578+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:45:51.538578+0000","last_peered":"2024-04-25T00:45:51.538578+0000","last_clean":"2024-04-25T00:45:51.538578+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:45:51.538578+0000","last_undegraded":"2024-04-25T00:45:51.538578+0000","last_fullsized":"2024-04-25T00:45:51.538578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'17","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598995+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:45:35.598995+0000","last_peered":"2024-04-25T00:45:35.598995+0000","last_clean":"2024-04-25T00:45:35.598995+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:45:35.598995+0000","last_undegraded":"2024-04-25T00:45:35.598995+0000","last_fullsized":"2024-04-25T00:45:35.598995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.547835+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:45:51.547835+0000","last_peered":"2024-04-25T00:45:51.547835+0000","last_clean":"2024-04-25T00:45:51.547835+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:45:51.547835+0000","last_undegraded":"2024-04-25T00:45:51.547835+0000","last_fullsized":"2024-04-25T00:45:51.547835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'35","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.021801+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:45:39.021801+0000","last_peered":"2024-04-25T00:45:39.021801+0000","last_clean":"2024-04-25T00:45:39.021801+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:45:39.021801+0000","last_undegraded":"2024-04-25T00:45:39.021801+0000","last_fullsized":"2024-04-25T00:45:39.021801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542493+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:45:51.542493+0000","last_peered":"2024-04-25T00:45:51.542493+0000","last_clean":"2024-04-25T00:45:51.542493+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:45:51.542493+0000","last_undegraded":"2024-04-25T00:45:51.542493+0000","last_fullsized":"2024-04-25T00:45:51.542493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'30","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:43.055262+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:45:43.055262+0000","last_peered":"2024-04-25T00:45:43.055262+0000","last_clean":"2024-04-25T00:45:43.055262+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:45:43.055262+0000","last_undegraded":"2024-04-25T00:45:43.055262+0000","last_fullsized":"2024-04-25T00:45:43.055262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.507789+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:45:51.507789+0000","last_peered":"2024-04-25T00:45:51.507789+0000","last_clean":"2024-04-25T00:45:51.507789+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:45:51.507789+0000","last_undegraded":"2024-04-25T00:45:51.507789+0000","last_fullsized":"2024-04-25T00:45:51.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'33","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:46.654507+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:45:46.654507+0000","last_peered":"2024-04-25T00:45:46.654507+0000","last_clean":"2024-04-25T00:45:46.654507+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:45:46.654507+0000","last_undegraded":"2024-04-25T00:45:46.654507+0000","last_fullsized":"2024-04-25T00:45:46.654507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538189+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:45:51.538189+0000","last_peered":"2024-04-25T00:45:51.538189+0000","last_clean":"2024-04-25T00:45:51.538189+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:45:51.538189+0000","last_undegraded":"2024-04-25T00:45:51.538189+0000","last_fullsized":"2024-04-25T00:45:51.538189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.597426+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:45:35.597426+0000","last_peered":"2024-04-25T00:45:35.597426+0000","last_clean":"2024-04-25T00:45:35.597426+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:45:35.597426+0000","last_undegraded":"2024-04-25T00:45:35.597426+0000","last_fullsized":"2024-04-25T00:45:35.597426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'39","reported_seq":56,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538292+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:45:51.538292+0000","last_peered":"2024-04-25T00:45:51.538292+0000","last_clean":"2024-04-25T00:45:51.538292+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:45:51.538292+0000","last_undegraded":"2024-04-25T00:45:51.538292+0000","last_fullsized":"2024-04-25T00:45:51.538292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.602720+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:45:35.602720+0000","last_peered":"2024-04-25T00:45:35.602720+0000","last_clean":"2024-04-25T00:45:35.602720+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:45:35.602720+0000","last_undegraded":"2024-04-25T00:45:35.602720+0000","last_fullsized":"2024-04-25T00:45:35.602720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.541141+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:45:51.541141+0000","last_peered":"2024-04-25T00:45:51.541141+0000","last_clean":"2024-04-25T00:45:51.541141+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:45:51.541141+0000","last_undegraded":"2024-04-25T00:45:51.541141+0000","last_fullsized":"2024-04-25T00:45:51.541141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.603187+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:45:35.603187+0000","last_peered":"2024-04-25T00:45:35.603187+0000","last_clean":"2024-04-25T00:45:35.603187+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:45:35.603187+0000","last_undegraded":"2024-04-25T00:45:35.603187+0000","last_fullsized":"2024-04-25T00:45:35.603187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'43","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526806+0000","last_change":"2024-04-25T00:43:39.828979+0000","last_active":"2024-04-25T00:45:51.526806+0000","last_peered":"2024-04-25T00:45:51.526806+0000","last_clean":"2024-04-25T00:45:51.526806+0000","last_became_active":"2024-04-25T00:43:39.827957+0000","last_became_peered":"2024-04-25T00:43:39.827957+0000","last_unstale":"2024-04-25T00:45:51.526806+0000","last_undegraded":"2024-04-25T00:45:51.526806+0000","last_fullsized":"2024-04-25T00:45:51.526806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:31.750071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'29","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598652+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:45:35.598652+0000","last_peered":"2024-04-25T00:45:35.598652+0000","last_clean":"2024-04-25T00:45:35.598652+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:45:35.598652+0000","last_undegraded":"2024-04-25T00:45:35.598652+0000","last_fullsized":"2024-04-25T00:45:35.598652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'46","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521754+0000","last_change":"2024-04-25T00:43:39.822064+0000","last_active":"2024-04-25T00:45:51.521754+0000","last_peered":"2024-04-25T00:45:51.521754+0000","last_clean":"2024-04-25T00:45:51.521754+0000","last_became_active":"2024-04-25T00:43:39.821411+0000","last_became_peered":"2024-04-25T00:43:39.821411+0000","last_unstale":"2024-04-25T00:45:51.521754+0000","last_undegraded":"2024-04-25T00:45:51.521754+0000","last_fullsized":"2024-04-25T00:45:51.521754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:43:39.818302+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:39.818302+0000","last_peered":"2024-04-25T00:43:39.818302+0000","last_clean":"2024-04-25T00:43:39.818302+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:39.818302+0000","last_undegraded":"2024-04-25T00:43:39.818302+0000","last_fullsized":"2024-04-25T00:43:39.818302+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.590322+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:45:35.590322+0000","last_peered":"2024-04-25T00:45:35.590322+0000","last_clean":"2024-04-25T00:45:35.590322+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:45:35.590322+0000","last_undegraded":"2024-04-25T00:45:35.590322+0000","last_fullsized":"2024-04-25T00:45:35.590322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'57","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527964+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:45:51.527964+0000","last_peered":"2024-04-25T00:45:51.527964+0000","last_clean":"2024-04-25T00:45:51.527964+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:45:51.527964+0000","last_undegraded":"2024-04-25T00:45:51.527964+0000","last_fullsized":"2024-04-25T00:45:51.527964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.604044+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:45:35.604044+0000","last_peered":"2024-04-25T00:45:35.604044+0000","last_clean":"2024-04-25T00:45:35.604044+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:45:35.604044+0000","last_undegraded":"2024-04-25T00:45:35.604044+0000","last_fullsized":"2024-04-25T00:45:35.604044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515969+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:45:51.515969+0000","last_peered":"2024-04-25T00:45:51.515969+0000","last_clean":"2024-04-25T00:45:51.515969+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:45:51.515969+0000","last_undegraded":"2024-04-25T00:45:51.515969+0000","last_fullsized":"2024-04-25T00:45:51.515969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'94","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:52.118906+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:45:52.118906+0000","last_peered":"2024-04-25T00:45:52.118906+0000","last_clean":"2024-04-25T00:45:52.118906+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:45:52.118906+0000","last_undegraded":"2024-04-25T00:45:52.118906+0000","last_fullsized":"2024-04-25T00:45:52.118906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357237,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":65,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515761+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:45:51.515761+0000","last_peered":"2024-04-25T00:45:51.515761+0000","last_clean":"2024-04-25T00:45:51.515761+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:45:51.515761+0000","last_undegraded":"2024-04-25T00:45:51.515761+0000","last_fullsized":"2024-04-25T00:45:51.515761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599954+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:45:35.599954+0000","last_peered":"2024-04-25T00:45:35.599954+0000","last_clean":"2024-04-25T00:45:35.599954+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:45:35.599954+0000","last_undegraded":"2024-04-25T00:45:35.599954+0000","last_fullsized":"2024-04-25T00:45:35.599954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522092+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:45:51.522092+0000","last_peered":"2024-04-25T00:45:51.522092+0000","last_clean":"2024-04-25T00:45:51.522092+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:45:51.522092+0000","last_undegraded":"2024-04-25T00:45:51.522092+0000","last_fullsized":"2024-04-25T00:45:51.522092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'22","reported_seq":41,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588232+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:45:35.588232+0000","last_peered":"2024-04-25T00:45:35.588232+0000","last_clean":"2024-04-25T00:45:35.588232+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:45:35.588232+0000","last_undegraded":"2024-04-25T00:45:35.588232+0000","last_fullsized":"2024-04-25T00:45:35.588232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533188+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:45:51.533188+0000","last_peered":"2024-04-25T00:45:51.533188+0000","last_clean":"2024-04-25T00:45:51.533188+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:45:51.533188+0000","last_undegraded":"2024-04-25T00:45:51.533188+0000","last_fullsized":"2024-04-25T00:45:51.533188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589938+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:45:35.589938+0000","last_peered":"2024-04-25T00:45:35.589938+0000","last_clean":"2024-04-25T00:45:35.589938+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:45:35.589938+0000","last_undegraded":"2024-04-25T00:45:35.589938+0000","last_fullsized":"2024-04-25T00:45:35.589938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516160+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:45:51.516160+0000","last_peered":"2024-04-25T00:45:51.516160+0000","last_clean":"2024-04-25T00:45:51.516160+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:45:51.516160+0000","last_undegraded":"2024-04-25T00:45:51.516160+0000","last_fullsized":"2024-04-25T00:45:51.516160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'94","reported_seq":113,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.752292+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:45:50.752292+0000","last_peered":"2024-04-25T00:45:50.752292+0000","last_clean":"2024-04-25T00:45:50.752292+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:45:50.752292+0000","last_undegraded":"2024-04-25T00:45:50.752292+0000","last_fullsized":"2024-04-25T00:45:50.752292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.496801+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:45:51.496801+0000","last_peered":"2024-04-25T00:45:51.496801+0000","last_clean":"2024-04-25T00:45:51.496801+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:45:51.496801+0000","last_undegraded":"2024-04-25T00:45:51.496801+0000","last_fullsized":"2024-04-25T00:45:51.496801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'26","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:49.162485+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:45:49.162485+0000","last_peered":"2024-04-25T00:45:49.162485+0000","last_clean":"2024-04-25T00:45:49.162485+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:45:49.162485+0000","last_undegraded":"2024-04-25T00:45:49.162485+0000","last_fullsized":"2024-04-25T00:45:49.162485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514084+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:45:51.514084+0000","last_peered":"2024-04-25T00:45:51.514084+0000","last_clean":"2024-04-25T00:45:51.514084+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:45:51.514084+0000","last_undegraded":"2024-04-25T00:45:51.514084+0000","last_fullsized":"2024-04-25T00:45:51.514084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599183+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:45:35.599183+0000","last_peered":"2024-04-25T00:45:35.599183+0000","last_clean":"2024-04-25T00:45:35.599183+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:45:35.599183+0000","last_undegraded":"2024-04-25T00:45:35.599183+0000","last_fullsized":"2024-04-25T00:45:35.599183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491953+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:45:51.491953+0000","last_peered":"2024-04-25T00:45:51.491953+0000","last_clean":"2024-04-25T00:45:51.491953+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:45:51.491953+0000","last_undegraded":"2024-04-25T00:45:51.491953+0000","last_fullsized":"2024-04-25T00:45:51.491953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:45.167958+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:45:45.167958+0000","last_peered":"2024-04-25T00:45:45.167958+0000","last_clean":"2024-04-25T00:45:45.167958+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:45:45.167958+0000","last_undegraded":"2024-04-25T00:45:45.167958+0000","last_fullsized":"2024-04-25T00:45:45.167958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510268+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:45:51.510268+0000","last_peered":"2024-04-25T00:45:51.510268+0000","last_clean":"2024-04-25T00:45:51.510268+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:45:51.510268+0000","last_undegraded":"2024-04-25T00:45:51.510268+0000","last_fullsized":"2024-04-25T00:45:51.510268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'31","reported_seq":50,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.592850+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:45:35.592850+0000","last_peered":"2024-04-25T00:45:35.592850+0000","last_clean":"2024-04-25T00:45:35.592850+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:45:35.592850+0000","last_undegraded":"2024-04-25T00:45:35.592850+0000","last_fullsized":"2024-04-25T00:45:35.592850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521036+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:45:51.521036+0000","last_peered":"2024-04-25T00:45:51.521036+0000","last_clean":"2024-04-25T00:45:51.521036+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:45:51.521036+0000","last_undegraded":"2024-04-25T00:45:51.521036+0000","last_fullsized":"2024-04-25T00:45:51.521036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'508","reported_seq":527,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.577934+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:45:35.577934+0000","last_peered":"2024-04-25T00:45:35.577934+0000","last_clean":"2024-04-25T00:45:35.577934+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:45:35.577934+0000","last_undegraded":"2024-04-25T00:45:35.577934+0000","last_fullsized":"2024-04-25T00:45:35.577934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.508442+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:45:51.508442+0000","last_peered":"2024-04-25T00:45:51.508442+0000","last_clean":"2024-04-25T00:45:51.508442+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:45:51.508442+0000","last_undegraded":"2024-04-25T00:45:51.508442+0000","last_fullsized":"2024-04-25T00:45:51.508442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.535096+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:45:51.535096+0000","last_peered":"2024-04-25T00:45:51.535096+0000","last_clean":"2024-04-25T00:45:51.535096+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:45:51.535096+0000","last_undegraded":"2024-04-25T00:45:51.535096+0000","last_fullsized":"2024-04-25T00:45:51.535096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'16","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.973394+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:45:32.973394+0000","last_peered":"2024-04-25T00:45:32.973394+0000","last_clean":"2024-04-25T00:45:32.973394+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:45:32.973394+0000","last_undegraded":"2024-04-25T00:45:32.973394+0000","last_fullsized":"2024-04-25T00:45:32.973394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533744+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:45:51.533744+0000","last_peered":"2024-04-25T00:45:51.533744+0000","last_clean":"2024-04-25T00:45:51.533744+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:45:51.533744+0000","last_undegraded":"2024-04-25T00:45:51.533744+0000","last_fullsized":"2024-04-25T00:45:51.533744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'62","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:03.994450+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:46:03.994450+0000","last_peered":"2024-04-25T00:46:03.994450+0000","last_clean":"2024-04-25T00:46:03.994450+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:46:03.994450+0000","last_undegraded":"2024-04-25T00:46:03.994450+0000","last_fullsized":"2024-04-25T00:46:03.994450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5442977,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":5338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'57","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509543+0000","last_change":"2024-04-25T00:43:39.829056+0000","last_active":"2024-04-25T00:45:51.509543+0000","last_peered":"2024-04-25T00:45:51.509543+0000","last_clean":"2024-04-25T00:45:51.509543+0000","last_became_active":"2024-04-25T00:43:39.828089+0000","last_became_peered":"2024-04-25T00:43:39.828089+0000","last_unstale":"2024-04-25T00:45:51.509543+0000","last_undegraded":"2024-04-25T00:45:51.509543+0000","last_fullsized":"2024-04-25T00:45:51.509543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:30.063490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'34","reported_seq":53,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.708112+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:45:39.708112+0000","last_peered":"2024-04-25T00:45:39.708112+0000","last_clean":"2024-04-25T00:45:39.708112+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:45:39.708112+0000","last_undegraded":"2024-04-25T00:45:39.708112+0000","last_fullsized":"2024-04-25T00:45:39.708112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510036+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:45:51.510036+0000","last_peered":"2024-04-25T00:45:51.510036+0000","last_clean":"2024-04-25T00:45:51.510036+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:45:51.510036+0000","last_undegraded":"2024-04-25T00:45:51.510036+0000","last_fullsized":"2024-04-25T00:45:51.510036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'48","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.581829+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:45:35.581829+0000","last_peered":"2024-04-25T00:45:35.581829+0000","last_clean":"2024-04-25T00:45:35.581829+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:45:35.581829+0000","last_undegraded":"2024-04-25T00:45:35.581829+0000","last_fullsized":"2024-04-25T00:45:35.581829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":4111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3307,"ondisk_log_size":3307,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":30967508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":3252,"num_write_kb":31581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61997056,"data_stored":61935016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3892,"ondisk_log_size":3892,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57908,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94313932,"statfs":{"total":96636764160,"available":96577466368,"internally_reserved":0,"allocated":18853888,"data_stored":18700145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14069760,"data_stored":13915563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":83396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39604,"kb_used_data":8428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332236,"statfs":{"total":96636764160,"available":96596209664,"internally_reserved":0,"allocated":8630272,"data_stored":8472040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52368,"kb_used_data":12872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319472,"statfs":{"total":96636764160,"available":96583139328,"internally_reserved":0,"allocated":13180928,"data_stored":13024127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":589824,"data_stored":440543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":83466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":9224192,"data_stored":9062920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":357237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12951552,"data_stored":12940821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8388734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13840384,"data_stored":13832257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18030592,"data_stored":18026471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:46:14.376 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:46:14.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fe6be640 1 Processor -- start 2024-04-25T00:46:14.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fe6be640 1 -- start start 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fe6be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51f81627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fe6be640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f8162cf0 con 0x7f51f80b6bf0 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fe6be640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f815eac0 con 0x7f51f8144590 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fe6be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51f815f6d0 con 0x7f51f81441b0 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fcc5b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f51f80b6bf0 0x7f51f8141ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42038/0 (socket says 172.21.15.129:42038) 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fcc5b640 1 -- 172.21.15.129:0/1877827208 learned_addr learned my addr 172.21.15.129:0/1877827208 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f7fff640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51f81627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 177929622 0 0) 0x7f51e80014a0 con 0x7f51f80b6bf0 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e0003530 con 0x7f51f80b6bf0 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3926862617 0 0) 0x7f51e0003530 con 0x7f51f80b6bf0 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 msgr2=0x7f51f81627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51f81627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 >> v1:172.21.15.196:6789/0 conn(0x7f51f8144590 legacy=0x7f51f80b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f815eda0 con 0x7f51f80b6bf0 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f51e8001bd0 con 0x7f51f80b6bf0 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 >> v1:172.21.15.129:6789/0 conn(0x7f51f80b6bf0 legacy=0x7f51f8141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51e00039f0 0x7f51e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51e0004810 0x7f51e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f51f815f6d0 con 0x7f51e0004810 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51e000ba80 con 0x7f51e00039f0 2024-04-25T00:46:14.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51e000bc80 con 0x7f51f81441b0 2024-04-25T00:46:14.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f77fe640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51e0004810 0x7f51e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f7fff640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51e00039f0 0x7f51e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fcc5b640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f77fe640 1 -- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 msgr2=0x7f51e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f77fe640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f77fe640 1 -- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51e00039f0 msgr2=0x7f51e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f77fe640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51e00039f0 0x7f51e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51f77fe640 1 -- 172.21.15.129:0/1877827208 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e0003530 con 0x7f51e0004810 2024-04-25T00:46:14.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.531+0000 7f51fcc5b640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51f81441b0 0x7f51e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:14.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51f77fe640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51e0004810 0x7f51e0004c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51ec001750 tx=0x7f51ec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec53623afb62190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0013a0 con 0x7f51e0004810 2024-04-25T00:46:14.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec0052f0 con 0x7f51e0004810 2024-04-25T00:46:14.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51f6ffd640 1 -- 172.21.15.129:0/1877827208 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec003840 con 0x7f51e0004810 2024-04-25T00:46:14.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 -- 172.21.15.129:0/1877827208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51e0004810 msgr2=0x7f51e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 --2- 172.21.15.129:0/1877827208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51e0004810 0x7f51e0004c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51ec001750 tx=0x7f51ec01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fcc5b640 1 -- 172.21.15.129:0/1877827208 reap_dead start 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 -- 172.21.15.129:0/1877827208 shutdown_connections 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 -- 172.21.15.129:0/1877827208 >> 172.21.15.129:0/1877827208 conn(0x7f51f80b52d0 msgr2=0x7f51f80b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 -- 172.21.15.129:0/1877827208 shutdown_connections 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 -- 172.21.15.129:0/1877827208 wait complete. 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 Processor -- start 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.535+0000 7f51fe6be640 1 -- start start 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51e0004810 0x7f51f8156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f8159940 0x7f51f81f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 0x7f51f81544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f51f815e760 con 0x7f51f8154060 2024-04-25T00:46:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51f815f360 con 0x7f51f8159940 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f51f8143a90 con 0x7f51e0004810 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 0x7f51f81544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 0x7f51f81544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38136/0 (socket says 172.21.15.129:38136) 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 -- 172.21.15.129:0/2182335238 learned_addr learned my addr 172.21.15.129:0/2182335238 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f7fff640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51e0004810 0x7f51f8156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f77fe640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f8159940 0x7f51f81f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 -- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51e0004810 msgr2=0x7f51f8156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51e0004810 0x7f51f8156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 -- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f8159940 msgr2=0x7f51f81f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f8159940 0x7f51f81f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 -- 172.21.15.129:0/2182335238 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f8157170 con 0x7f51f8154060 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fcc5b640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 0x7f51f81544c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51f80b34e0 tx=0x7f51e8010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e801a070 con 0x7f51f8154060 2024-04-25T00:46:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e8011040 con 0x7f51f8154060 2024-04-25T00:46:14.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f8157450 con 0x7f51f8154060 2024-04-25T00:46:14.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e800eb50 con 0x7f51f8154060 2024-04-25T00:46:14.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f8157a10 con 0x7f51f8154060 2024-04-25T00:46:14.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f8062e50 con 0x7f51f8154060 2024-04-25T00:46:14.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f51e801d070 con 0x7f51f8154060 2024-04-25T00:46:14.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f4ff9640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51e40678f0 0x7f51e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f7fff640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51e40678f0 0x7f51e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f51e80884b0 con 0x7f51f8154060 2024-04-25T00:46:14.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.539+0000 7f51f7fff640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51e40678f0 0x7f51e4069db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009a30 tx=0x7f51f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.543+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f51f8162cf0 con 0x7f51f8154060 2024-04-25T00:46:14.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.703+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51f8141550 con 0x7f51e40678f0 2024-04-25T00:46:14.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.719+0000 7f51f4ff9640 1 -- 172.21.15.129:0/2182335238 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f51f8141550 con 0x7f51e40678f0 2024-04-25T00:46:14.723 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:14.726 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51e40678f0 msgr2=0x7f51e4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51e40678f0 0x7f51e4069db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009a30 tx=0x7f51f0008040 comp rx=0 tx=0).stop 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 msgr2=0x7f51f81544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 0x7f51f81544c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51f80b34e0 tx=0x7f51e8010c30 comp rx=0 tx=0).stop 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 shutdown_connections 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f51e40678f0 0x7f51e4069db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51f8154060 0x7f51f81544c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51f8159940 0x7f51f81f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 --2- 172.21.15.129:0/2182335238 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51e0004810 0x7f51f8156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 >> 172.21.15.129:0/2182335238 conn(0x7f51f80b52d0 msgr2=0x7f51f815fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 shutdown_connections 2024-04-25T00:46:14.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.723+0000 7f51fe6be640 1 -- 172.21.15.129:0/2182335238 wait complete. 2024-04-25T00:46:14.744 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-25T00:46:14.664685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31557876,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":181,"num_read_kb":491,"num_write":3411,"num_write_kb":33997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7316,"ondisk_log_size":7316,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333704,"kb_used_data":63496,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754641016,"statfs":{"total":773094113280,"available":772752400384,"internally_reserved":0,"allocated":65019904,"data_stored":63782200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004055"},"pg_stats":[{"pgid":"3.3f","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.546368+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:45:51.546368+0000","last_peered":"2024-04-25T00:45:51.546368+0000","last_clean":"2024-04-25T00:45:51.546368+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:45:51.546368+0000","last_undegraded":"2024-04-25T00:45:51.546368+0000","last_fullsized":"2024-04-25T00:45:51.546368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491744+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:45:51.491744+0000","last_peered":"2024-04-25T00:45:51.491744+0000","last_clean":"2024-04-25T00:45:51.491744+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:45:51.491744+0000","last_undegraded":"2024-04-25T00:45:51.491744+0000","last_fullsized":"2024-04-25T00:45:51.491744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515512+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:45:51.515512+0000","last_peered":"2024-04-25T00:45:51.515512+0000","last_clean":"2024-04-25T00:45:51.515512+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:45:51.515512+0000","last_undegraded":"2024-04-25T00:45:51.515512+0000","last_fullsized":"2024-04-25T00:45:51.515512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'40","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527087+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:45:51.527087+0000","last_peered":"2024-04-25T00:45:51.527087+0000","last_clean":"2024-04-25T00:45:51.527087+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:45:51.527087+0000","last_undegraded":"2024-04-25T00:45:51.527087+0000","last_fullsized":"2024-04-25T00:45:51.527087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.539538+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:45:51.539538+0000","last_peered":"2024-04-25T00:45:51.539538+0000","last_clean":"2024-04-25T00:45:51.539538+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:45:51.539538+0000","last_undegraded":"2024-04-25T00:45:51.539538+0000","last_fullsized":"2024-04-25T00:45:51.539538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527080+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:45:51.527080+0000","last_peered":"2024-04-25T00:45:51.527080+0000","last_clean":"2024-04-25T00:45:51.527080+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:45:51.527080+0000","last_undegraded":"2024-04-25T00:45:51.527080+0000","last_fullsized":"2024-04-25T00:45:51.527080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.545654+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:45:51.545654+0000","last_peered":"2024-04-25T00:45:51.545654+0000","last_clean":"2024-04-25T00:45:51.545654+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:45:51.545654+0000","last_undegraded":"2024-04-25T00:45:51.545654+0000","last_fullsized":"2024-04-25T00:45:51.545654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525802+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:45:51.525802+0000","last_peered":"2024-04-25T00:45:51.525802+0000","last_clean":"2024-04-25T00:45:51.525802+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:45:51.525802+0000","last_undegraded":"2024-04-25T00:45:51.525802+0000","last_fullsized":"2024-04-25T00:45:51.525802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'54","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542405+0000","last_change":"2024-04-25T00:43:39.829539+0000","last_active":"2024-04-25T00:45:51.542405+0000","last_peered":"2024-04-25T00:45:51.542405+0000","last_clean":"2024-04-25T00:45:51.542405+0000","last_became_active":"2024-04-25T00:43:39.828586+0000","last_became_peered":"2024-04-25T00:43:39.828586+0000","last_unstale":"2024-04-25T00:45:51.542405+0000","last_undegraded":"2024-04-25T00:45:51.542405+0000","last_fullsized":"2024-04-25T00:45:51.542405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:31.663001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.540595+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:45:51.540595+0000","last_peered":"2024-04-25T00:45:51.540595+0000","last_clean":"2024-04-25T00:45:51.540595+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:45:51.540595+0000","last_undegraded":"2024-04-25T00:45:51.540595+0000","last_fullsized":"2024-04-25T00:45:51.540595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.528273+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:45:51.528273+0000","last_peered":"2024-04-25T00:45:51.528273+0000","last_clean":"2024-04-25T00:45:51.528273+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:45:51.528273+0000","last_undegraded":"2024-04-25T00:45:51.528273+0000","last_fullsized":"2024-04-25T00:45:51.528273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'44","reported_seq":61,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.532319+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:45:51.532319+0000","last_peered":"2024-04-25T00:45:51.532319+0000","last_clean":"2024-04-25T00:45:51.532319+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:45:51.532319+0000","last_undegraded":"2024-04-25T00:45:51.532319+0000","last_fullsized":"2024-04-25T00:45:51.532319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.512541+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:45:51.512541+0000","last_peered":"2024-04-25T00:45:51.512541+0000","last_clean":"2024-04-25T00:45:51.512541+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:45:51.512541+0000","last_undegraded":"2024-04-25T00:45:51.512541+0000","last_fullsized":"2024-04-25T00:45:51.512541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.519079+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:45:51.519079+0000","last_peered":"2024-04-25T00:45:51.519079+0000","last_clean":"2024-04-25T00:45:51.519079+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:45:51.519079+0000","last_undegraded":"2024-04-25T00:45:51.519079+0000","last_fullsized":"2024-04-25T00:45:51.519079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.495151+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:45:51.495151+0000","last_peered":"2024-04-25T00:45:51.495151+0000","last_clean":"2024-04-25T00:45:51.495151+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:45:51.495151+0000","last_undegraded":"2024-04-25T00:45:51.495151+0000","last_fullsized":"2024-04-25T00:45:51.495151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522025+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:45:51.522025+0000","last_peered":"2024-04-25T00:45:51.522025+0000","last_clean":"2024-04-25T00:45:51.522025+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:45:51.522025+0000","last_undegraded":"2024-04-25T00:45:51.522025+0000","last_fullsized":"2024-04-25T00:45:51.522025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549462+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:45:51.549462+0000","last_peered":"2024-04-25T00:45:51.549462+0000","last_clean":"2024-04-25T00:45:51.549462+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:45:51.549462+0000","last_undegraded":"2024-04-25T00:45:51.549462+0000","last_fullsized":"2024-04-25T00:45:51.549462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.531867+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:45:51.531867+0000","last_peered":"2024-04-25T00:45:51.531867+0000","last_clean":"2024-04-25T00:45:51.531867+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:45:51.531867+0000","last_undegraded":"2024-04-25T00:45:51.531867+0000","last_fullsized":"2024-04-25T00:45:51.531867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533095+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:45:51.533095+0000","last_peered":"2024-04-25T00:45:51.533095+0000","last_clean":"2024-04-25T00:45:51.533095+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:45:51.533095+0000","last_undegraded":"2024-04-25T00:45:51.533095+0000","last_fullsized":"2024-04-25T00:45:51.533095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'62","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515861+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:45:51.515861+0000","last_peered":"2024-04-25T00:45:51.515861+0000","last_clean":"2024-04-25T00:45:51.515861+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:45:51.515861+0000","last_undegraded":"2024-04-25T00:45:51.515861+0000","last_fullsized":"2024-04-25T00:45:51.515861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548225+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:45:51.548225+0000","last_peered":"2024-04-25T00:45:51.548225+0000","last_clean":"2024-04-25T00:45:51.548225+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:45:51.548225+0000","last_undegraded":"2024-04-25T00:45:51.548225+0000","last_fullsized":"2024-04-25T00:45:51.548225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549483+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:45:51.549483+0000","last_peered":"2024-04-25T00:45:51.549483+0000","last_clean":"2024-04-25T00:45:51.549483+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:45:51.549483+0000","last_undegraded":"2024-04-25T00:45:51.549483+0000","last_fullsized":"2024-04-25T00:45:51.549483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510069+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:45:51.510069+0000","last_peered":"2024-04-25T00:45:51.510069+0000","last_clean":"2024-04-25T00:45:51.510069+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:45:51.510069+0000","last_undegraded":"2024-04-25T00:45:51.510069+0000","last_fullsized":"2024-04-25T00:45:51.510069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526220+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:45:51.526220+0000","last_peered":"2024-04-25T00:45:51.526220+0000","last_clean":"2024-04-25T00:45:51.526220+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:45:51.526220+0000","last_undegraded":"2024-04-25T00:45:51.526220+0000","last_fullsized":"2024-04-25T00:45:51.526220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.499400+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:45:51.499400+0000","last_peered":"2024-04-25T00:45:51.499400+0000","last_clean":"2024-04-25T00:45:51.499400+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:45:51.499400+0000","last_undegraded":"2024-04-25T00:45:51.499400+0000","last_fullsized":"2024-04-25T00:45:51.499400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'66","reported_seq":83,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548312+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:45:51.548312+0000","last_peered":"2024-04-25T00:45:51.548312+0000","last_clean":"2024-04-25T00:45:51.548312+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:45:51.548312+0000","last_undegraded":"2024-04-25T00:45:51.548312+0000","last_fullsized":"2024-04-25T00:45:51.548312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'37","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500102+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:45:51.500102+0000","last_peered":"2024-04-25T00:45:51.500102+0000","last_clean":"2024-04-25T00:45:51.500102+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:45:51.500102+0000","last_undegraded":"2024-04-25T00:45:51.500102+0000","last_fullsized":"2024-04-25T00:45:51.500102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509134+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:45:51.509134+0000","last_peered":"2024-04-25T00:45:51.509134+0000","last_clean":"2024-04-25T00:45:51.509134+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:45:51.509134+0000","last_undegraded":"2024-04-25T00:45:51.509134+0000","last_fullsized":"2024-04-25T00:45:51.509134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'52","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525040+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:45:51.525040+0000","last_peered":"2024-04-25T00:45:51.525040+0000","last_clean":"2024-04-25T00:45:51.525040+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:45:51.525040+0000","last_undegraded":"2024-04-25T00:45:51.525040+0000","last_fullsized":"2024-04-25T00:45:51.525040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'58","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500399+0000","last_change":"2024-04-25T00:43:39.818085+0000","last_active":"2024-04-25T00:45:51.500399+0000","last_peered":"2024-04-25T00:45:51.500399+0000","last_clean":"2024-04-25T00:45:51.500399+0000","last_became_active":"2024-04-25T00:43:39.817067+0000","last_became_peered":"2024-04-25T00:43:39.817067+0000","last_unstale":"2024-04-25T00:45:51.500399+0000","last_undegraded":"2024-04-25T00:45:51.500399+0000","last_fullsized":"2024-04-25T00:45:51.500399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.520841+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:45:51.520841+0000","last_peered":"2024-04-25T00:45:51.520841+0000","last_clean":"2024-04-25T00:45:51.520841+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:45:51.520841+0000","last_undegraded":"2024-04-25T00:45:51.520841+0000","last_fullsized":"2024-04-25T00:45:51.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'69","reported_seq":86,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515490+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:45:51.515490+0000","last_peered":"2024-04-25T00:45:51.515490+0000","last_clean":"2024-04-25T00:45:51.515490+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:45:51.515490+0000","last_undegraded":"2024-04-25T00:45:51.515490+0000","last_fullsized":"2024-04-25T00:45:51.515490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'44","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514482+0000","last_change":"2024-04-25T00:43:39.825468+0000","last_active":"2024-04-25T00:45:51.514482+0000","last_peered":"2024-04-25T00:45:51.514482+0000","last_clean":"2024-04-25T00:45:51.514482+0000","last_became_active":"2024-04-25T00:43:39.824614+0000","last_became_peered":"2024-04-25T00:43:39.824614+0000","last_unstale":"2024-04-25T00:45:51.514482+0000","last_undegraded":"2024-04-25T00:45:51.514482+0000","last_fullsized":"2024-04-25T00:45:51.514482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1500","reported_seq":1519,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.578349+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:45:35.578349+0000","last_peered":"2024-04-25T00:45:35.578349+0000","last_clean":"2024-04-25T00:45:35.578349+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:45:35.578349+0000","last_undegraded":"2024-04-25T00:45:35.578349+0000","last_fullsized":"2024-04-25T00:45:35.578349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1500,"log_dups_size":0,"ondisk_log_size":1500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1474,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.530748+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:45:51.530748+0000","last_peered":"2024-04-25T00:45:51.530748+0000","last_clean":"2024-04-25T00:45:51.530748+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:45:51.530748+0000","last_undegraded":"2024-04-25T00:45:51.530748+0000","last_fullsized":"2024-04-25T00:45:51.530748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'36","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588170+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:45:35.588170+0000","last_peered":"2024-04-25T00:45:35.588170+0000","last_clean":"2024-04-25T00:45:35.588170+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:45:35.588170+0000","last_undegraded":"2024-04-25T00:45:35.588170+0000","last_fullsized":"2024-04-25T00:45:35.588170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'19","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.962699+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:45:32.962699+0000","last_peered":"2024-04-25T00:45:32.962699+0000","last_clean":"2024-04-25T00:45:32.962699+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:45:32.962699+0000","last_undegraded":"2024-04-25T00:45:32.962699+0000","last_fullsized":"2024-04-25T00:45:32.962699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'65","reported_seq":82,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516989+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:45:51.516989+0000","last_peered":"2024-04-25T00:45:51.516989+0000","last_clean":"2024-04-25T00:45:51.516989+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:45:51.516989+0000","last_undegraded":"2024-04-25T00:45:51.516989+0000","last_fullsized":"2024-04-25T00:45:51.516989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589858+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:45:35.589858+0000","last_peered":"2024-04-25T00:45:35.589858+0000","last_clean":"2024-04-25T00:45:35.589858+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:45:35.589858+0000","last_undegraded":"2024-04-25T00:45:35.589858+0000","last_fullsized":"2024-04-25T00:45:35.589858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.504832+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:45:51.504832+0000","last_peered":"2024-04-25T00:45:51.504832+0000","last_clean":"2024-04-25T00:45:51.504832+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:45:51.504832+0000","last_undegraded":"2024-04-25T00:45:51.504832+0000","last_fullsized":"2024-04-25T00:45:51.504832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.595446+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:45:35.595446+0000","last_peered":"2024-04-25T00:45:35.595446+0000","last_clean":"2024-04-25T00:45:35.595446+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:45:35.595446+0000","last_undegraded":"2024-04-25T00:45:35.595446+0000","last_fullsized":"2024-04-25T00:45:35.595446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'73","reported_seq":90,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549377+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:45:51.549377+0000","last_peered":"2024-04-25T00:45:51.549377+0000","last_clean":"2024-04-25T00:45:51.549377+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:45:51.549377+0000","last_undegraded":"2024-04-25T00:45:51.549377+0000","last_fullsized":"2024-04-25T00:45:51.549377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599552+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:45:35.599552+0000","last_peered":"2024-04-25T00:45:35.599552+0000","last_clean":"2024-04-25T00:45:35.599552+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:45:35.599552+0000","last_undegraded":"2024-04-25T00:45:35.599552+0000","last_fullsized":"2024-04-25T00:45:35.599552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521602+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:45:51.521602+0000","last_peered":"2024-04-25T00:45:51.521602+0000","last_clean":"2024-04-25T00:45:51.521602+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:45:51.521602+0000","last_undegraded":"2024-04-25T00:45:51.521602+0000","last_fullsized":"2024-04-25T00:45:51.521602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'55","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.142036+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:45:50.142036+0000","last_peered":"2024-04-25T00:45:50.142036+0000","last_clean":"2024-04-25T00:45:50.142036+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:45:50.142036+0000","last_undegraded":"2024-04-25T00:45:50.142036+0000","last_fullsized":"2024-04-25T00:45:50.142036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526491+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:45:51.526491+0000","last_peered":"2024-04-25T00:45:51.526491+0000","last_clean":"2024-04-25T00:45:51.526491+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:45:51.526491+0000","last_undegraded":"2024-04-25T00:45:51.526491+0000","last_fullsized":"2024-04-25T00:45:51.526491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'927","reported_seq":946,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:47.871752+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:45:47.871752+0000","last_peered":"2024-04-25T00:45:47.871752+0000","last_clean":"2024-04-25T00:45:47.871752+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:45:47.871752+0000","last_undegraded":"2024-04-25T00:45:47.871752+0000","last_fullsized":"2024-04-25T00:45:47.871752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":908,"num_write_kb":4392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538578+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:45:51.538578+0000","last_peered":"2024-04-25T00:45:51.538578+0000","last_clean":"2024-04-25T00:45:51.538578+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:45:51.538578+0000","last_undegraded":"2024-04-25T00:45:51.538578+0000","last_fullsized":"2024-04-25T00:45:51.538578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'17","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598995+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:45:35.598995+0000","last_peered":"2024-04-25T00:45:35.598995+0000","last_clean":"2024-04-25T00:45:35.598995+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:45:35.598995+0000","last_undegraded":"2024-04-25T00:45:35.598995+0000","last_fullsized":"2024-04-25T00:45:35.598995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.547835+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:45:51.547835+0000","last_peered":"2024-04-25T00:45:51.547835+0000","last_clean":"2024-04-25T00:45:51.547835+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:45:51.547835+0000","last_undegraded":"2024-04-25T00:45:51.547835+0000","last_fullsized":"2024-04-25T00:45:51.547835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'35","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.021801+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:45:39.021801+0000","last_peered":"2024-04-25T00:45:39.021801+0000","last_clean":"2024-04-25T00:45:39.021801+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:45:39.021801+0000","last_undegraded":"2024-04-25T00:45:39.021801+0000","last_fullsized":"2024-04-25T00:45:39.021801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542493+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:45:51.542493+0000","last_peered":"2024-04-25T00:45:51.542493+0000","last_clean":"2024-04-25T00:45:51.542493+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:45:51.542493+0000","last_undegraded":"2024-04-25T00:45:51.542493+0000","last_fullsized":"2024-04-25T00:45:51.542493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'30","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:43.055262+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:45:43.055262+0000","last_peered":"2024-04-25T00:45:43.055262+0000","last_clean":"2024-04-25T00:45:43.055262+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:45:43.055262+0000","last_undegraded":"2024-04-25T00:45:43.055262+0000","last_fullsized":"2024-04-25T00:45:43.055262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.507789+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:45:51.507789+0000","last_peered":"2024-04-25T00:45:51.507789+0000","last_clean":"2024-04-25T00:45:51.507789+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:45:51.507789+0000","last_undegraded":"2024-04-25T00:45:51.507789+0000","last_fullsized":"2024-04-25T00:45:51.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'33","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:46.654507+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:45:46.654507+0000","last_peered":"2024-04-25T00:45:46.654507+0000","last_clean":"2024-04-25T00:45:46.654507+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:45:46.654507+0000","last_undegraded":"2024-04-25T00:45:46.654507+0000","last_fullsized":"2024-04-25T00:45:46.654507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538189+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:45:51.538189+0000","last_peered":"2024-04-25T00:45:51.538189+0000","last_clean":"2024-04-25T00:45:51.538189+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:45:51.538189+0000","last_undegraded":"2024-04-25T00:45:51.538189+0000","last_fullsized":"2024-04-25T00:45:51.538189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.597426+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:45:35.597426+0000","last_peered":"2024-04-25T00:45:35.597426+0000","last_clean":"2024-04-25T00:45:35.597426+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:45:35.597426+0000","last_undegraded":"2024-04-25T00:45:35.597426+0000","last_fullsized":"2024-04-25T00:45:35.597426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'39","reported_seq":56,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538292+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:45:51.538292+0000","last_peered":"2024-04-25T00:45:51.538292+0000","last_clean":"2024-04-25T00:45:51.538292+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:45:51.538292+0000","last_undegraded":"2024-04-25T00:45:51.538292+0000","last_fullsized":"2024-04-25T00:45:51.538292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.602720+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:45:35.602720+0000","last_peered":"2024-04-25T00:45:35.602720+0000","last_clean":"2024-04-25T00:45:35.602720+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:45:35.602720+0000","last_undegraded":"2024-04-25T00:45:35.602720+0000","last_fullsized":"2024-04-25T00:45:35.602720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.541141+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:45:51.541141+0000","last_peered":"2024-04-25T00:45:51.541141+0000","last_clean":"2024-04-25T00:45:51.541141+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:45:51.541141+0000","last_undegraded":"2024-04-25T00:45:51.541141+0000","last_fullsized":"2024-04-25T00:45:51.541141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.603187+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:45:35.603187+0000","last_peered":"2024-04-25T00:45:35.603187+0000","last_clean":"2024-04-25T00:45:35.603187+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:45:35.603187+0000","last_undegraded":"2024-04-25T00:45:35.603187+0000","last_fullsized":"2024-04-25T00:45:35.603187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'43","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526806+0000","last_change":"2024-04-25T00:43:39.828979+0000","last_active":"2024-04-25T00:45:51.526806+0000","last_peered":"2024-04-25T00:45:51.526806+0000","last_clean":"2024-04-25T00:45:51.526806+0000","last_became_active":"2024-04-25T00:43:39.827957+0000","last_became_peered":"2024-04-25T00:43:39.827957+0000","last_unstale":"2024-04-25T00:45:51.526806+0000","last_undegraded":"2024-04-25T00:45:51.526806+0000","last_fullsized":"2024-04-25T00:45:51.526806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:31.750071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'29","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598652+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:45:35.598652+0000","last_peered":"2024-04-25T00:45:35.598652+0000","last_clean":"2024-04-25T00:45:35.598652+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:45:35.598652+0000","last_undegraded":"2024-04-25T00:45:35.598652+0000","last_fullsized":"2024-04-25T00:45:35.598652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'46","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521754+0000","last_change":"2024-04-25T00:43:39.822064+0000","last_active":"2024-04-25T00:45:51.521754+0000","last_peered":"2024-04-25T00:45:51.521754+0000","last_clean":"2024-04-25T00:45:51.521754+0000","last_became_active":"2024-04-25T00:43:39.821411+0000","last_became_peered":"2024-04-25T00:43:39.821411+0000","last_unstale":"2024-04-25T00:45:51.521754+0000","last_undegraded":"2024-04-25T00:45:51.521754+0000","last_fullsized":"2024-04-25T00:45:51.521754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:43:39.818302+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:39.818302+0000","last_peered":"2024-04-25T00:43:39.818302+0000","last_clean":"2024-04-25T00:43:39.818302+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:39.818302+0000","last_undegraded":"2024-04-25T00:43:39.818302+0000","last_fullsized":"2024-04-25T00:43:39.818302+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.590322+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:45:35.590322+0000","last_peered":"2024-04-25T00:45:35.590322+0000","last_clean":"2024-04-25T00:45:35.590322+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:45:35.590322+0000","last_undegraded":"2024-04-25T00:45:35.590322+0000","last_fullsized":"2024-04-25T00:45:35.590322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'57","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527964+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:45:51.527964+0000","last_peered":"2024-04-25T00:45:51.527964+0000","last_clean":"2024-04-25T00:45:51.527964+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:45:51.527964+0000","last_undegraded":"2024-04-25T00:45:51.527964+0000","last_fullsized":"2024-04-25T00:45:51.527964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.604044+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:45:35.604044+0000","last_peered":"2024-04-25T00:45:35.604044+0000","last_clean":"2024-04-25T00:45:35.604044+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:45:35.604044+0000","last_undegraded":"2024-04-25T00:45:35.604044+0000","last_fullsized":"2024-04-25T00:45:35.604044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515969+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:45:51.515969+0000","last_peered":"2024-04-25T00:45:51.515969+0000","last_clean":"2024-04-25T00:45:51.515969+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:45:51.515969+0000","last_undegraded":"2024-04-25T00:45:51.515969+0000","last_fullsized":"2024-04-25T00:45:51.515969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'94","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:52.118906+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:45:52.118906+0000","last_peered":"2024-04-25T00:45:52.118906+0000","last_clean":"2024-04-25T00:45:52.118906+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:45:52.118906+0000","last_undegraded":"2024-04-25T00:45:52.118906+0000","last_fullsized":"2024-04-25T00:45:52.118906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357237,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":65,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515761+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:45:51.515761+0000","last_peered":"2024-04-25T00:45:51.515761+0000","last_clean":"2024-04-25T00:45:51.515761+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:45:51.515761+0000","last_undegraded":"2024-04-25T00:45:51.515761+0000","last_fullsized":"2024-04-25T00:45:51.515761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599954+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:45:35.599954+0000","last_peered":"2024-04-25T00:45:35.599954+0000","last_clean":"2024-04-25T00:45:35.599954+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:45:35.599954+0000","last_undegraded":"2024-04-25T00:45:35.599954+0000","last_fullsized":"2024-04-25T00:45:35.599954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522092+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:45:51.522092+0000","last_peered":"2024-04-25T00:45:51.522092+0000","last_clean":"2024-04-25T00:45:51.522092+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:45:51.522092+0000","last_undegraded":"2024-04-25T00:45:51.522092+0000","last_fullsized":"2024-04-25T00:45:51.522092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'22","reported_seq":41,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588232+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:45:35.588232+0000","last_peered":"2024-04-25T00:45:35.588232+0000","last_clean":"2024-04-25T00:45:35.588232+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:45:35.588232+0000","last_undegraded":"2024-04-25T00:45:35.588232+0000","last_fullsized":"2024-04-25T00:45:35.588232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533188+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:45:51.533188+0000","last_peered":"2024-04-25T00:45:51.533188+0000","last_clean":"2024-04-25T00:45:51.533188+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:45:51.533188+0000","last_undegraded":"2024-04-25T00:45:51.533188+0000","last_fullsized":"2024-04-25T00:45:51.533188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589938+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:45:35.589938+0000","last_peered":"2024-04-25T00:45:35.589938+0000","last_clean":"2024-04-25T00:45:35.589938+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:45:35.589938+0000","last_undegraded":"2024-04-25T00:45:35.589938+0000","last_fullsized":"2024-04-25T00:45:35.589938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516160+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:45:51.516160+0000","last_peered":"2024-04-25T00:45:51.516160+0000","last_clean":"2024-04-25T00:45:51.516160+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:45:51.516160+0000","last_undegraded":"2024-04-25T00:45:51.516160+0000","last_fullsized":"2024-04-25T00:45:51.516160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'94","reported_seq":113,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.752292+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:45:50.752292+0000","last_peered":"2024-04-25T00:45:50.752292+0000","last_clean":"2024-04-25T00:45:50.752292+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:45:50.752292+0000","last_undegraded":"2024-04-25T00:45:50.752292+0000","last_fullsized":"2024-04-25T00:45:50.752292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.496801+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:45:51.496801+0000","last_peered":"2024-04-25T00:45:51.496801+0000","last_clean":"2024-04-25T00:45:51.496801+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:45:51.496801+0000","last_undegraded":"2024-04-25T00:45:51.496801+0000","last_fullsized":"2024-04-25T00:45:51.496801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'26","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:49.162485+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:45:49.162485+0000","last_peered":"2024-04-25T00:45:49.162485+0000","last_clean":"2024-04-25T00:45:49.162485+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:45:49.162485+0000","last_undegraded":"2024-04-25T00:45:49.162485+0000","last_fullsized":"2024-04-25T00:45:49.162485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514084+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:45:51.514084+0000","last_peered":"2024-04-25T00:45:51.514084+0000","last_clean":"2024-04-25T00:45:51.514084+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:45:51.514084+0000","last_undegraded":"2024-04-25T00:45:51.514084+0000","last_fullsized":"2024-04-25T00:45:51.514084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599183+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:45:35.599183+0000","last_peered":"2024-04-25T00:45:35.599183+0000","last_clean":"2024-04-25T00:45:35.599183+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:45:35.599183+0000","last_undegraded":"2024-04-25T00:45:35.599183+0000","last_fullsized":"2024-04-25T00:45:35.599183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491953+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:45:51.491953+0000","last_peered":"2024-04-25T00:45:51.491953+0000","last_clean":"2024-04-25T00:45:51.491953+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:45:51.491953+0000","last_undegraded":"2024-04-25T00:45:51.491953+0000","last_fullsized":"2024-04-25T00:45:51.491953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:45.167958+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:45:45.167958+0000","last_peered":"2024-04-25T00:45:45.167958+0000","last_clean":"2024-04-25T00:45:45.167958+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:45:45.167958+0000","last_undegraded":"2024-04-25T00:45:45.167958+0000","last_fullsized":"2024-04-25T00:45:45.167958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510268+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:45:51.510268+0000","last_peered":"2024-04-25T00:45:51.510268+0000","last_clean":"2024-04-25T00:45:51.510268+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:45:51.510268+0000","last_undegraded":"2024-04-25T00:45:51.510268+0000","last_fullsized":"2024-04-25T00:45:51.510268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'31","reported_seq":50,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.592850+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:45:35.592850+0000","last_peered":"2024-04-25T00:45:35.592850+0000","last_clean":"2024-04-25T00:45:35.592850+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:45:35.592850+0000","last_undegraded":"2024-04-25T00:45:35.592850+0000","last_fullsized":"2024-04-25T00:45:35.592850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521036+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:45:51.521036+0000","last_peered":"2024-04-25T00:45:51.521036+0000","last_clean":"2024-04-25T00:45:51.521036+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:45:51.521036+0000","last_undegraded":"2024-04-25T00:45:51.521036+0000","last_fullsized":"2024-04-25T00:45:51.521036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'508","reported_seq":527,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.577934+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:45:35.577934+0000","last_peered":"2024-04-25T00:45:35.577934+0000","last_clean":"2024-04-25T00:45:35.577934+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:45:35.577934+0000","last_undegraded":"2024-04-25T00:45:35.577934+0000","last_fullsized":"2024-04-25T00:45:35.577934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.508442+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:45:51.508442+0000","last_peered":"2024-04-25T00:45:51.508442+0000","last_clean":"2024-04-25T00:45:51.508442+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:45:51.508442+0000","last_undegraded":"2024-04-25T00:45:51.508442+0000","last_fullsized":"2024-04-25T00:45:51.508442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.535096+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:45:51.535096+0000","last_peered":"2024-04-25T00:45:51.535096+0000","last_clean":"2024-04-25T00:45:51.535096+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:45:51.535096+0000","last_undegraded":"2024-04-25T00:45:51.535096+0000","last_fullsized":"2024-04-25T00:45:51.535096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'16","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.973394+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:45:32.973394+0000","last_peered":"2024-04-25T00:45:32.973394+0000","last_clean":"2024-04-25T00:45:32.973394+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:45:32.973394+0000","last_undegraded":"2024-04-25T00:45:32.973394+0000","last_fullsized":"2024-04-25T00:45:32.973394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533744+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:45:51.533744+0000","last_peered":"2024-04-25T00:45:51.533744+0000","last_clean":"2024-04-25T00:45:51.533744+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:45:51.533744+0000","last_undegraded":"2024-04-25T00:45:51.533744+0000","last_fullsized":"2024-04-25T00:45:51.533744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'62","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:03.994450+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:46:03.994450+0000","last_peered":"2024-04-25T00:46:03.994450+0000","last_clean":"2024-04-25T00:46:03.994450+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:46:03.994450+0000","last_undegraded":"2024-04-25T00:46:03.994450+0000","last_fullsized":"2024-04-25T00:46:03.994450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5442977,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":5338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'57","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509543+0000","last_change":"2024-04-25T00:43:39.829056+0000","last_active":"2024-04-25T00:45:51.509543+0000","last_peered":"2024-04-25T00:45:51.509543+0000","last_clean":"2024-04-25T00:45:51.509543+0000","last_became_active":"2024-04-25T00:43:39.828089+0000","last_became_peered":"2024-04-25T00:43:39.828089+0000","last_unstale":"2024-04-25T00:45:51.509543+0000","last_undegraded":"2024-04-25T00:45:51.509543+0000","last_fullsized":"2024-04-25T00:45:51.509543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:30.063490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'34","reported_seq":53,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.708112+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:45:39.708112+0000","last_peered":"2024-04-25T00:45:39.708112+0000","last_clean":"2024-04-25T00:45:39.708112+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:45:39.708112+0000","last_undegraded":"2024-04-25T00:45:39.708112+0000","last_fullsized":"2024-04-25T00:45:39.708112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510036+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:45:51.510036+0000","last_peered":"2024-04-25T00:45:51.510036+0000","last_clean":"2024-04-25T00:45:51.510036+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:45:51.510036+0000","last_undegraded":"2024-04-25T00:45:51.510036+0000","last_fullsized":"2024-04-25T00:45:51.510036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'48","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.581829+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:45:35.581829+0000","last_peered":"2024-04-25T00:45:35.581829+0000","last_clean":"2024-04-25T00:45:35.581829+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:45:35.581829+0000","last_undegraded":"2024-04-25T00:45:35.581829+0000","last_fullsized":"2024-04-25T00:45:35.581829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":4111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3307,"ondisk_log_size":3307,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":30967508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":3252,"num_write_kb":31581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61997056,"data_stored":61935016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3892,"ondisk_log_size":3892,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57908,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94313932,"statfs":{"total":96636764160,"available":96577466368,"internally_reserved":0,"allocated":18853888,"data_stored":18700145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14069760,"data_stored":13915563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":83396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39604,"kb_used_data":8428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332236,"statfs":{"total":96636764160,"available":96596209664,"internally_reserved":0,"allocated":8630272,"data_stored":8472040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52368,"kb_used_data":12872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319472,"statfs":{"total":96636764160,"available":96583139328,"internally_reserved":0,"allocated":13180928,"data_stored":13024127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":589824,"data_stored":440543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":83466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":9224192,"data_stored":9062920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":357237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12951552,"data_stored":12940821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8388734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13840384,"data_stored":13832257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18030592,"data_stored":18026471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:46:14.748 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:46:14.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:46:14.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.907+0000 7f2b74c34640 1 Processor -- start 2024-04-25T00:46:14.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.907+0000 7f2b74c34640 1 -- start start 2024-04-25T00:46:14.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b74c34640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 0x7f2b700723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b74c34640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b70072aa0 con 0x7f2b7015c7f0 2024-04-25T00:46:14.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b74c34640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b700708e0 con 0x7f2b700cd190 2024-04-25T00:46:14.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b74c34640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b70070690 con 0x7f2b700ccdb0 2024-04-25T00:46:14.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6dd9b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f2b7015c7f0 0x7f2b70158640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42050/0 (socket says 172.21.15.129:42050) 2024-04-25T00:46:14.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6dd9b640 1 -- 172.21.15.129:0/2152742958 learned_addr learned my addr 172.21.15.129:0/2152742958 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6e59c640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 0x7f2b700723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319067025 0 0) 0x7f2b70072aa0 con 0x7f2b7015c7f0 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b58003530 con 0x7f2b7015c7f0 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4282489105 0 0) 0x7f2b58003530 con 0x7f2b7015c7f0 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 msgr2=0x7f2b700723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 0x7f2b700723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 >> v1:172.21.15.196:6789/0 conn(0x7f2b700cd190 legacy=0x7f2b7006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b70070bc0 con 0x7f2b7015c7f0 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f2b64001680 con 0x7f2b7015c7f0 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 >> v1:172.21.15.129:6789/0 conn(0x7f2b7015c7f0 legacy=0x7f2b70158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b580039f0 0x7f2b58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 0x7f2b58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b58004d50 0x7f2b580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70070690 con 0x7f2b58004d50 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5800bb00 con 0x7f2b580039f0 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5800bd00 con 0x7f2b700ccdb0 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b58004d50 0x7f2b580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6e59c640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b580039f0 0x7f2b58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 -- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 msgr2=0x7f2b58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b700ccdb0 0x7f2b58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 -- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b580039f0 msgr2=0x7f2b58003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b580039f0 0x7f2b58003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 -- 172.21.15.129:0/2152742958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b58003530 con 0x7f2b58004d50 2024-04-25T00:46:14.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6ed9d640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b58004d50 0x7f2b580051d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b68001e70 tx=0x7f2b6801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93519d95d93db204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68001a90 con 0x7f2b58004d50 2024-04-25T00:46:14.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68003b80 con 0x7f2b58004d50 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.911+0000 7f2b6d59a640 1 -- 172.21.15.129:0/2152742958 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68005080 con 0x7f2b58004d50 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2152742958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b58004d50 msgr2=0x7f2b580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2152742958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b58004d50 0x7f2b580051d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b68001e70 tx=0x7f2b6801c780 comp rx=0 tx=0).stop 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6ed9d640 1 -- 172.21.15.129:0/2152742958 reap_dead start 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2152742958 shutdown_connections 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2152742958 >> 172.21.15.129:0/2152742958 conn(0x7f2b700b14a0 msgr2=0x7f2b700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2152742958 shutdown_connections 2024-04-25T00:46:14.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2152742958 wait complete. 2024-04-25T00:46:14.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 Processor -- start 2024-04-25T00:46:14.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- start start 2024-04-25T00:46:14.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b58004d50 0x7f2b701e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 0x7f2b7013c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7015ae20 0x7f2b70155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700723e0 con 0x7f2b701e4a20 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70158fe0 con 0x7f2b7015ae20 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b70072560 con 0x7f2b58004d50 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 0x7f2b7013c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 0x7f2b7013c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38156/0 (socket says 172.21.15.129:38156) 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 -- 172.21.15.129:0/2251849313 learned_addr learned my addr 172.21.15.129:0/2251849313 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6e59c640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b58004d50 0x7f2b701e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6ed9d640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7015ae20 0x7f2b70155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 -- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b58004d50 msgr2=0x7f2b701e33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b58004d50 0x7f2b701e33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 -- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7015ae20 msgr2=0x7f2b70155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7015ae20 0x7f2b70155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 -- 172.21.15.129:0/2251849313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7013cf30 con 0x7f2b701e4a20 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b6dd9b640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 0x7f2b7013c9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400f3a0 tx=0x7f2b6400f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64007d00 con 0x7f2b701e4a20 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6400a0d0 con 0x7f2b701e4a20 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400a540 con 0x7f2b701e4a20 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b701e3980 con 0x7f2b701e4a20 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.915+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b701e3f40 con 0x7f2b701e4a20 2024-04-25T00:46:14.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.919+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b70062e50 con 0x7f2b701e4a20 2024-04-25T00:46:14.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.919+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b64014070 con 0x7f2b701e4a20 2024-04-25T00:46:14.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.919+0000 7f2b5effd640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b500678f0 0x7f2b50069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:14.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.919+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f2b64088870 con 0x7f2b701e4a20 2024-04-25T00:46:14.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.919+0000 7f2b6e59c640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b500678f0 0x7f2b50069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:14.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.919+0000 7f2b6e59c640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b500678f0 0x7f2b50069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b701563e0 tx=0x7f2b6000b3a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:14.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:14.923+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2b6401c260 con 0x7f2b701e4a20 2024-04-25T00:46:15.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.067+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b701e4ea0 con 0x7f2b500678f0 2024-04-25T00:46:15.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.075+0000 7f2b5effd640 1 -- 172.21.15.129:0/2251849313 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f2b6005ea10 con 0x7f2b500678f0 2024-04-25T00:46:15.081 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:15.083 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b500678f0 msgr2=0x7f2b50069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b500678f0 0x7f2b50069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b701563e0 tx=0x7f2b6000b3a0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 msgr2=0x7f2b7013c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 0x7f2b7013c9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400f3a0 tx=0x7f2b6400f860 comp rx=0 tx=0).stop 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 shutdown_connections 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b500678f0 0x7f2b50069db0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b701563e0 tx=0x7f2b6000b3a0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b7015ae20 0x7f2b70155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b701e4a20 0x7f2b7013c9f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 --2- 172.21.15.129:0/2251849313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b58004d50 0x7f2b701e33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 >> 172.21.15.129:0/2251849313 conn(0x7f2b700b14a0 msgr2=0x7f2b7015ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:15.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 shutdown_connections 2024-04-25T00:46:15.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.079+0000 7f2b74c34640 1 -- 172.21.15.129:0/2251849313 wait complete. 2024-04-25T00:46:15.098 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-25T00:46:14.664685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31557876,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":181,"num_read_kb":491,"num_write":3411,"num_write_kb":33997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7316,"ondisk_log_size":7316,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333704,"kb_used_data":63496,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754641016,"statfs":{"total":773094113280,"available":772752400384,"internally_reserved":0,"allocated":65019904,"data_stored":63782200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004055"},"pg_stats":[{"pgid":"3.3f","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.546368+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:45:51.546368+0000","last_peered":"2024-04-25T00:45:51.546368+0000","last_clean":"2024-04-25T00:45:51.546368+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:45:51.546368+0000","last_undegraded":"2024-04-25T00:45:51.546368+0000","last_fullsized":"2024-04-25T00:45:51.546368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491744+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:45:51.491744+0000","last_peered":"2024-04-25T00:45:51.491744+0000","last_clean":"2024-04-25T00:45:51.491744+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:45:51.491744+0000","last_undegraded":"2024-04-25T00:45:51.491744+0000","last_fullsized":"2024-04-25T00:45:51.491744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515512+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:45:51.515512+0000","last_peered":"2024-04-25T00:45:51.515512+0000","last_clean":"2024-04-25T00:45:51.515512+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:45:51.515512+0000","last_undegraded":"2024-04-25T00:45:51.515512+0000","last_fullsized":"2024-04-25T00:45:51.515512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'40","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527087+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:45:51.527087+0000","last_peered":"2024-04-25T00:45:51.527087+0000","last_clean":"2024-04-25T00:45:51.527087+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:45:51.527087+0000","last_undegraded":"2024-04-25T00:45:51.527087+0000","last_fullsized":"2024-04-25T00:45:51.527087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.539538+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:45:51.539538+0000","last_peered":"2024-04-25T00:45:51.539538+0000","last_clean":"2024-04-25T00:45:51.539538+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:45:51.539538+0000","last_undegraded":"2024-04-25T00:45:51.539538+0000","last_fullsized":"2024-04-25T00:45:51.539538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527080+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:45:51.527080+0000","last_peered":"2024-04-25T00:45:51.527080+0000","last_clean":"2024-04-25T00:45:51.527080+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:45:51.527080+0000","last_undegraded":"2024-04-25T00:45:51.527080+0000","last_fullsized":"2024-04-25T00:45:51.527080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.545654+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:45:51.545654+0000","last_peered":"2024-04-25T00:45:51.545654+0000","last_clean":"2024-04-25T00:45:51.545654+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:45:51.545654+0000","last_undegraded":"2024-04-25T00:45:51.545654+0000","last_fullsized":"2024-04-25T00:45:51.545654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525802+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:45:51.525802+0000","last_peered":"2024-04-25T00:45:51.525802+0000","last_clean":"2024-04-25T00:45:51.525802+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:45:51.525802+0000","last_undegraded":"2024-04-25T00:45:51.525802+0000","last_fullsized":"2024-04-25T00:45:51.525802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'54","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542405+0000","last_change":"2024-04-25T00:43:39.829539+0000","last_active":"2024-04-25T00:45:51.542405+0000","last_peered":"2024-04-25T00:45:51.542405+0000","last_clean":"2024-04-25T00:45:51.542405+0000","last_became_active":"2024-04-25T00:43:39.828586+0000","last_became_peered":"2024-04-25T00:43:39.828586+0000","last_unstale":"2024-04-25T00:45:51.542405+0000","last_undegraded":"2024-04-25T00:45:51.542405+0000","last_fullsized":"2024-04-25T00:45:51.542405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:31.663001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.540595+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:45:51.540595+0000","last_peered":"2024-04-25T00:45:51.540595+0000","last_clean":"2024-04-25T00:45:51.540595+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:45:51.540595+0000","last_undegraded":"2024-04-25T00:45:51.540595+0000","last_fullsized":"2024-04-25T00:45:51.540595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.528273+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:45:51.528273+0000","last_peered":"2024-04-25T00:45:51.528273+0000","last_clean":"2024-04-25T00:45:51.528273+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:45:51.528273+0000","last_undegraded":"2024-04-25T00:45:51.528273+0000","last_fullsized":"2024-04-25T00:45:51.528273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'44","reported_seq":61,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.532319+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:45:51.532319+0000","last_peered":"2024-04-25T00:45:51.532319+0000","last_clean":"2024-04-25T00:45:51.532319+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:45:51.532319+0000","last_undegraded":"2024-04-25T00:45:51.532319+0000","last_fullsized":"2024-04-25T00:45:51.532319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.512541+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:45:51.512541+0000","last_peered":"2024-04-25T00:45:51.512541+0000","last_clean":"2024-04-25T00:45:51.512541+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:45:51.512541+0000","last_undegraded":"2024-04-25T00:45:51.512541+0000","last_fullsized":"2024-04-25T00:45:51.512541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.519079+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:45:51.519079+0000","last_peered":"2024-04-25T00:45:51.519079+0000","last_clean":"2024-04-25T00:45:51.519079+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:45:51.519079+0000","last_undegraded":"2024-04-25T00:45:51.519079+0000","last_fullsized":"2024-04-25T00:45:51.519079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.495151+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:45:51.495151+0000","last_peered":"2024-04-25T00:45:51.495151+0000","last_clean":"2024-04-25T00:45:51.495151+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:45:51.495151+0000","last_undegraded":"2024-04-25T00:45:51.495151+0000","last_fullsized":"2024-04-25T00:45:51.495151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522025+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:45:51.522025+0000","last_peered":"2024-04-25T00:45:51.522025+0000","last_clean":"2024-04-25T00:45:51.522025+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:45:51.522025+0000","last_undegraded":"2024-04-25T00:45:51.522025+0000","last_fullsized":"2024-04-25T00:45:51.522025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549462+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:45:51.549462+0000","last_peered":"2024-04-25T00:45:51.549462+0000","last_clean":"2024-04-25T00:45:51.549462+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:45:51.549462+0000","last_undegraded":"2024-04-25T00:45:51.549462+0000","last_fullsized":"2024-04-25T00:45:51.549462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.531867+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:45:51.531867+0000","last_peered":"2024-04-25T00:45:51.531867+0000","last_clean":"2024-04-25T00:45:51.531867+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:45:51.531867+0000","last_undegraded":"2024-04-25T00:45:51.531867+0000","last_fullsized":"2024-04-25T00:45:51.531867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533095+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:45:51.533095+0000","last_peered":"2024-04-25T00:45:51.533095+0000","last_clean":"2024-04-25T00:45:51.533095+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:45:51.533095+0000","last_undegraded":"2024-04-25T00:45:51.533095+0000","last_fullsized":"2024-04-25T00:45:51.533095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'62","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515861+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:45:51.515861+0000","last_peered":"2024-04-25T00:45:51.515861+0000","last_clean":"2024-04-25T00:45:51.515861+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:45:51.515861+0000","last_undegraded":"2024-04-25T00:45:51.515861+0000","last_fullsized":"2024-04-25T00:45:51.515861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548225+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:45:51.548225+0000","last_peered":"2024-04-25T00:45:51.548225+0000","last_clean":"2024-04-25T00:45:51.548225+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:45:51.548225+0000","last_undegraded":"2024-04-25T00:45:51.548225+0000","last_fullsized":"2024-04-25T00:45:51.548225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549483+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:45:51.549483+0000","last_peered":"2024-04-25T00:45:51.549483+0000","last_clean":"2024-04-25T00:45:51.549483+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:45:51.549483+0000","last_undegraded":"2024-04-25T00:45:51.549483+0000","last_fullsized":"2024-04-25T00:45:51.549483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510069+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:45:51.510069+0000","last_peered":"2024-04-25T00:45:51.510069+0000","last_clean":"2024-04-25T00:45:51.510069+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:45:51.510069+0000","last_undegraded":"2024-04-25T00:45:51.510069+0000","last_fullsized":"2024-04-25T00:45:51.510069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526220+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:45:51.526220+0000","last_peered":"2024-04-25T00:45:51.526220+0000","last_clean":"2024-04-25T00:45:51.526220+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:45:51.526220+0000","last_undegraded":"2024-04-25T00:45:51.526220+0000","last_fullsized":"2024-04-25T00:45:51.526220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.499400+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:45:51.499400+0000","last_peered":"2024-04-25T00:45:51.499400+0000","last_clean":"2024-04-25T00:45:51.499400+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:45:51.499400+0000","last_undegraded":"2024-04-25T00:45:51.499400+0000","last_fullsized":"2024-04-25T00:45:51.499400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'66","reported_seq":83,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548312+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:45:51.548312+0000","last_peered":"2024-04-25T00:45:51.548312+0000","last_clean":"2024-04-25T00:45:51.548312+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:45:51.548312+0000","last_undegraded":"2024-04-25T00:45:51.548312+0000","last_fullsized":"2024-04-25T00:45:51.548312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'37","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500102+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:45:51.500102+0000","last_peered":"2024-04-25T00:45:51.500102+0000","last_clean":"2024-04-25T00:45:51.500102+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:45:51.500102+0000","last_undegraded":"2024-04-25T00:45:51.500102+0000","last_fullsized":"2024-04-25T00:45:51.500102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509134+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:45:51.509134+0000","last_peered":"2024-04-25T00:45:51.509134+0000","last_clean":"2024-04-25T00:45:51.509134+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:45:51.509134+0000","last_undegraded":"2024-04-25T00:45:51.509134+0000","last_fullsized":"2024-04-25T00:45:51.509134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'52","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525040+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:45:51.525040+0000","last_peered":"2024-04-25T00:45:51.525040+0000","last_clean":"2024-04-25T00:45:51.525040+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:45:51.525040+0000","last_undegraded":"2024-04-25T00:45:51.525040+0000","last_fullsized":"2024-04-25T00:45:51.525040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'58","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500399+0000","last_change":"2024-04-25T00:43:39.818085+0000","last_active":"2024-04-25T00:45:51.500399+0000","last_peered":"2024-04-25T00:45:51.500399+0000","last_clean":"2024-04-25T00:45:51.500399+0000","last_became_active":"2024-04-25T00:43:39.817067+0000","last_became_peered":"2024-04-25T00:43:39.817067+0000","last_unstale":"2024-04-25T00:45:51.500399+0000","last_undegraded":"2024-04-25T00:45:51.500399+0000","last_fullsized":"2024-04-25T00:45:51.500399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.520841+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:45:51.520841+0000","last_peered":"2024-04-25T00:45:51.520841+0000","last_clean":"2024-04-25T00:45:51.520841+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:45:51.520841+0000","last_undegraded":"2024-04-25T00:45:51.520841+0000","last_fullsized":"2024-04-25T00:45:51.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'69","reported_seq":86,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515490+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:45:51.515490+0000","last_peered":"2024-04-25T00:45:51.515490+0000","last_clean":"2024-04-25T00:45:51.515490+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:45:51.515490+0000","last_undegraded":"2024-04-25T00:45:51.515490+0000","last_fullsized":"2024-04-25T00:45:51.515490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'44","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514482+0000","last_change":"2024-04-25T00:43:39.825468+0000","last_active":"2024-04-25T00:45:51.514482+0000","last_peered":"2024-04-25T00:45:51.514482+0000","last_clean":"2024-04-25T00:45:51.514482+0000","last_became_active":"2024-04-25T00:43:39.824614+0000","last_became_peered":"2024-04-25T00:43:39.824614+0000","last_unstale":"2024-04-25T00:45:51.514482+0000","last_undegraded":"2024-04-25T00:45:51.514482+0000","last_fullsized":"2024-04-25T00:45:51.514482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1500","reported_seq":1519,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.578349+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:45:35.578349+0000","last_peered":"2024-04-25T00:45:35.578349+0000","last_clean":"2024-04-25T00:45:35.578349+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:45:35.578349+0000","last_undegraded":"2024-04-25T00:45:35.578349+0000","last_fullsized":"2024-04-25T00:45:35.578349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1500,"log_dups_size":0,"ondisk_log_size":1500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1474,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.530748+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:45:51.530748+0000","last_peered":"2024-04-25T00:45:51.530748+0000","last_clean":"2024-04-25T00:45:51.530748+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:45:51.530748+0000","last_undegraded":"2024-04-25T00:45:51.530748+0000","last_fullsized":"2024-04-25T00:45:51.530748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'36","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588170+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:45:35.588170+0000","last_peered":"2024-04-25T00:45:35.588170+0000","last_clean":"2024-04-25T00:45:35.588170+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:45:35.588170+0000","last_undegraded":"2024-04-25T00:45:35.588170+0000","last_fullsized":"2024-04-25T00:45:35.588170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'19","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.962699+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:45:32.962699+0000","last_peered":"2024-04-25T00:45:32.962699+0000","last_clean":"2024-04-25T00:45:32.962699+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:45:32.962699+0000","last_undegraded":"2024-04-25T00:45:32.962699+0000","last_fullsized":"2024-04-25T00:45:32.962699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'65","reported_seq":82,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516989+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:45:51.516989+0000","last_peered":"2024-04-25T00:45:51.516989+0000","last_clean":"2024-04-25T00:45:51.516989+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:45:51.516989+0000","last_undegraded":"2024-04-25T00:45:51.516989+0000","last_fullsized":"2024-04-25T00:45:51.516989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589858+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:45:35.589858+0000","last_peered":"2024-04-25T00:45:35.589858+0000","last_clean":"2024-04-25T00:45:35.589858+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:45:35.589858+0000","last_undegraded":"2024-04-25T00:45:35.589858+0000","last_fullsized":"2024-04-25T00:45:35.589858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.504832+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:45:51.504832+0000","last_peered":"2024-04-25T00:45:51.504832+0000","last_clean":"2024-04-25T00:45:51.504832+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:45:51.504832+0000","last_undegraded":"2024-04-25T00:45:51.504832+0000","last_fullsized":"2024-04-25T00:45:51.504832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.595446+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:45:35.595446+0000","last_peered":"2024-04-25T00:45:35.595446+0000","last_clean":"2024-04-25T00:45:35.595446+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:45:35.595446+0000","last_undegraded":"2024-04-25T00:45:35.595446+0000","last_fullsized":"2024-04-25T00:45:35.595446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'73","reported_seq":90,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549377+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:45:51.549377+0000","last_peered":"2024-04-25T00:45:51.549377+0000","last_clean":"2024-04-25T00:45:51.549377+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:45:51.549377+0000","last_undegraded":"2024-04-25T00:45:51.549377+0000","last_fullsized":"2024-04-25T00:45:51.549377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599552+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:45:35.599552+0000","last_peered":"2024-04-25T00:45:35.599552+0000","last_clean":"2024-04-25T00:45:35.599552+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:45:35.599552+0000","last_undegraded":"2024-04-25T00:45:35.599552+0000","last_fullsized":"2024-04-25T00:45:35.599552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521602+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:45:51.521602+0000","last_peered":"2024-04-25T00:45:51.521602+0000","last_clean":"2024-04-25T00:45:51.521602+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:45:51.521602+0000","last_undegraded":"2024-04-25T00:45:51.521602+0000","last_fullsized":"2024-04-25T00:45:51.521602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'55","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.142036+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:45:50.142036+0000","last_peered":"2024-04-25T00:45:50.142036+0000","last_clean":"2024-04-25T00:45:50.142036+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:45:50.142036+0000","last_undegraded":"2024-04-25T00:45:50.142036+0000","last_fullsized":"2024-04-25T00:45:50.142036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526491+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:45:51.526491+0000","last_peered":"2024-04-25T00:45:51.526491+0000","last_clean":"2024-04-25T00:45:51.526491+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:45:51.526491+0000","last_undegraded":"2024-04-25T00:45:51.526491+0000","last_fullsized":"2024-04-25T00:45:51.526491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'927","reported_seq":946,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:47.871752+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:45:47.871752+0000","last_peered":"2024-04-25T00:45:47.871752+0000","last_clean":"2024-04-25T00:45:47.871752+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:45:47.871752+0000","last_undegraded":"2024-04-25T00:45:47.871752+0000","last_fullsized":"2024-04-25T00:45:47.871752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":908,"num_write_kb":4392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538578+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:45:51.538578+0000","last_peered":"2024-04-25T00:45:51.538578+0000","last_clean":"2024-04-25T00:45:51.538578+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:45:51.538578+0000","last_undegraded":"2024-04-25T00:45:51.538578+0000","last_fullsized":"2024-04-25T00:45:51.538578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'17","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598995+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:45:35.598995+0000","last_peered":"2024-04-25T00:45:35.598995+0000","last_clean":"2024-04-25T00:45:35.598995+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:45:35.598995+0000","last_undegraded":"2024-04-25T00:45:35.598995+0000","last_fullsized":"2024-04-25T00:45:35.598995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.547835+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:45:51.547835+0000","last_peered":"2024-04-25T00:45:51.547835+0000","last_clean":"2024-04-25T00:45:51.547835+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:45:51.547835+0000","last_undegraded":"2024-04-25T00:45:51.547835+0000","last_fullsized":"2024-04-25T00:45:51.547835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'35","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.021801+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:45:39.021801+0000","last_peered":"2024-04-25T00:45:39.021801+0000","last_clean":"2024-04-25T00:45:39.021801+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:45:39.021801+0000","last_undegraded":"2024-04-25T00:45:39.021801+0000","last_fullsized":"2024-04-25T00:45:39.021801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542493+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:45:51.542493+0000","last_peered":"2024-04-25T00:45:51.542493+0000","last_clean":"2024-04-25T00:45:51.542493+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:45:51.542493+0000","last_undegraded":"2024-04-25T00:45:51.542493+0000","last_fullsized":"2024-04-25T00:45:51.542493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'30","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:43.055262+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:45:43.055262+0000","last_peered":"2024-04-25T00:45:43.055262+0000","last_clean":"2024-04-25T00:45:43.055262+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:45:43.055262+0000","last_undegraded":"2024-04-25T00:45:43.055262+0000","last_fullsized":"2024-04-25T00:45:43.055262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.507789+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:45:51.507789+0000","last_peered":"2024-04-25T00:45:51.507789+0000","last_clean":"2024-04-25T00:45:51.507789+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:45:51.507789+0000","last_undegraded":"2024-04-25T00:45:51.507789+0000","last_fullsized":"2024-04-25T00:45:51.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'33","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:46.654507+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:45:46.654507+0000","last_peered":"2024-04-25T00:45:46.654507+0000","last_clean":"2024-04-25T00:45:46.654507+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:45:46.654507+0000","last_undegraded":"2024-04-25T00:45:46.654507+0000","last_fullsized":"2024-04-25T00:45:46.654507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538189+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:45:51.538189+0000","last_peered":"2024-04-25T00:45:51.538189+0000","last_clean":"2024-04-25T00:45:51.538189+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:45:51.538189+0000","last_undegraded":"2024-04-25T00:45:51.538189+0000","last_fullsized":"2024-04-25T00:45:51.538189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.597426+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:45:35.597426+0000","last_peered":"2024-04-25T00:45:35.597426+0000","last_clean":"2024-04-25T00:45:35.597426+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:45:35.597426+0000","last_undegraded":"2024-04-25T00:45:35.597426+0000","last_fullsized":"2024-04-25T00:45:35.597426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'39","reported_seq":56,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538292+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:45:51.538292+0000","last_peered":"2024-04-25T00:45:51.538292+0000","last_clean":"2024-04-25T00:45:51.538292+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:45:51.538292+0000","last_undegraded":"2024-04-25T00:45:51.538292+0000","last_fullsized":"2024-04-25T00:45:51.538292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.602720+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:45:35.602720+0000","last_peered":"2024-04-25T00:45:35.602720+0000","last_clean":"2024-04-25T00:45:35.602720+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:45:35.602720+0000","last_undegraded":"2024-04-25T00:45:35.602720+0000","last_fullsized":"2024-04-25T00:45:35.602720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.541141+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:45:51.541141+0000","last_peered":"2024-04-25T00:45:51.541141+0000","last_clean":"2024-04-25T00:45:51.541141+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:45:51.541141+0000","last_undegraded":"2024-04-25T00:45:51.541141+0000","last_fullsized":"2024-04-25T00:45:51.541141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.603187+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:45:35.603187+0000","last_peered":"2024-04-25T00:45:35.603187+0000","last_clean":"2024-04-25T00:45:35.603187+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:45:35.603187+0000","last_undegraded":"2024-04-25T00:45:35.603187+0000","last_fullsized":"2024-04-25T00:45:35.603187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'43","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526806+0000","last_change":"2024-04-25T00:43:39.828979+0000","last_active":"2024-04-25T00:45:51.526806+0000","last_peered":"2024-04-25T00:45:51.526806+0000","last_clean":"2024-04-25T00:45:51.526806+0000","last_became_active":"2024-04-25T00:43:39.827957+0000","last_became_peered":"2024-04-25T00:43:39.827957+0000","last_unstale":"2024-04-25T00:45:51.526806+0000","last_undegraded":"2024-04-25T00:45:51.526806+0000","last_fullsized":"2024-04-25T00:45:51.526806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:31.750071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'29","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598652+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:45:35.598652+0000","last_peered":"2024-04-25T00:45:35.598652+0000","last_clean":"2024-04-25T00:45:35.598652+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:45:35.598652+0000","last_undegraded":"2024-04-25T00:45:35.598652+0000","last_fullsized":"2024-04-25T00:45:35.598652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'46","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521754+0000","last_change":"2024-04-25T00:43:39.822064+0000","last_active":"2024-04-25T00:45:51.521754+0000","last_peered":"2024-04-25T00:45:51.521754+0000","last_clean":"2024-04-25T00:45:51.521754+0000","last_became_active":"2024-04-25T00:43:39.821411+0000","last_became_peered":"2024-04-25T00:43:39.821411+0000","last_unstale":"2024-04-25T00:45:51.521754+0000","last_undegraded":"2024-04-25T00:45:51.521754+0000","last_fullsized":"2024-04-25T00:45:51.521754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:43:39.818302+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:39.818302+0000","last_peered":"2024-04-25T00:43:39.818302+0000","last_clean":"2024-04-25T00:43:39.818302+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:39.818302+0000","last_undegraded":"2024-04-25T00:43:39.818302+0000","last_fullsized":"2024-04-25T00:43:39.818302+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.590322+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:45:35.590322+0000","last_peered":"2024-04-25T00:45:35.590322+0000","last_clean":"2024-04-25T00:45:35.590322+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:45:35.590322+0000","last_undegraded":"2024-04-25T00:45:35.590322+0000","last_fullsized":"2024-04-25T00:45:35.590322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'57","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527964+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:45:51.527964+0000","last_peered":"2024-04-25T00:45:51.527964+0000","last_clean":"2024-04-25T00:45:51.527964+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:45:51.527964+0000","last_undegraded":"2024-04-25T00:45:51.527964+0000","last_fullsized":"2024-04-25T00:45:51.527964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.604044+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:45:35.604044+0000","last_peered":"2024-04-25T00:45:35.604044+0000","last_clean":"2024-04-25T00:45:35.604044+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:45:35.604044+0000","last_undegraded":"2024-04-25T00:45:35.604044+0000","last_fullsized":"2024-04-25T00:45:35.604044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515969+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:45:51.515969+0000","last_peered":"2024-04-25T00:45:51.515969+0000","last_clean":"2024-04-25T00:45:51.515969+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:45:51.515969+0000","last_undegraded":"2024-04-25T00:45:51.515969+0000","last_fullsized":"2024-04-25T00:45:51.515969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'94","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:52.118906+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:45:52.118906+0000","last_peered":"2024-04-25T00:45:52.118906+0000","last_clean":"2024-04-25T00:45:52.118906+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:45:52.118906+0000","last_undegraded":"2024-04-25T00:45:52.118906+0000","last_fullsized":"2024-04-25T00:45:52.118906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357237,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":65,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515761+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:45:51.515761+0000","last_peered":"2024-04-25T00:45:51.515761+0000","last_clean":"2024-04-25T00:45:51.515761+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:45:51.515761+0000","last_undegraded":"2024-04-25T00:45:51.515761+0000","last_fullsized":"2024-04-25T00:45:51.515761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599954+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:45:35.599954+0000","last_peered":"2024-04-25T00:45:35.599954+0000","last_clean":"2024-04-25T00:45:35.599954+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:45:35.599954+0000","last_undegraded":"2024-04-25T00:45:35.599954+0000","last_fullsized":"2024-04-25T00:45:35.599954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522092+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:45:51.522092+0000","last_peered":"2024-04-25T00:45:51.522092+0000","last_clean":"2024-04-25T00:45:51.522092+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:45:51.522092+0000","last_undegraded":"2024-04-25T00:45:51.522092+0000","last_fullsized":"2024-04-25T00:45:51.522092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'22","reported_seq":41,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588232+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:45:35.588232+0000","last_peered":"2024-04-25T00:45:35.588232+0000","last_clean":"2024-04-25T00:45:35.588232+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:45:35.588232+0000","last_undegraded":"2024-04-25T00:45:35.588232+0000","last_fullsized":"2024-04-25T00:45:35.588232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533188+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:45:51.533188+0000","last_peered":"2024-04-25T00:45:51.533188+0000","last_clean":"2024-04-25T00:45:51.533188+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:45:51.533188+0000","last_undegraded":"2024-04-25T00:45:51.533188+0000","last_fullsized":"2024-04-25T00:45:51.533188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589938+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:45:35.589938+0000","last_peered":"2024-04-25T00:45:35.589938+0000","last_clean":"2024-04-25T00:45:35.589938+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:45:35.589938+0000","last_undegraded":"2024-04-25T00:45:35.589938+0000","last_fullsized":"2024-04-25T00:45:35.589938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516160+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:45:51.516160+0000","last_peered":"2024-04-25T00:45:51.516160+0000","last_clean":"2024-04-25T00:45:51.516160+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:45:51.516160+0000","last_undegraded":"2024-04-25T00:45:51.516160+0000","last_fullsized":"2024-04-25T00:45:51.516160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'94","reported_seq":113,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.752292+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:45:50.752292+0000","last_peered":"2024-04-25T00:45:50.752292+0000","last_clean":"2024-04-25T00:45:50.752292+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:45:50.752292+0000","last_undegraded":"2024-04-25T00:45:50.752292+0000","last_fullsized":"2024-04-25T00:45:50.752292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.496801+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:45:51.496801+0000","last_peered":"2024-04-25T00:45:51.496801+0000","last_clean":"2024-04-25T00:45:51.496801+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:45:51.496801+0000","last_undegraded":"2024-04-25T00:45:51.496801+0000","last_fullsized":"2024-04-25T00:45:51.496801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'26","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:49.162485+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:45:49.162485+0000","last_peered":"2024-04-25T00:45:49.162485+0000","last_clean":"2024-04-25T00:45:49.162485+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:45:49.162485+0000","last_undegraded":"2024-04-25T00:45:49.162485+0000","last_fullsized":"2024-04-25T00:45:49.162485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514084+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:45:51.514084+0000","last_peered":"2024-04-25T00:45:51.514084+0000","last_clean":"2024-04-25T00:45:51.514084+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:45:51.514084+0000","last_undegraded":"2024-04-25T00:45:51.514084+0000","last_fullsized":"2024-04-25T00:45:51.514084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599183+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:45:35.599183+0000","last_peered":"2024-04-25T00:45:35.599183+0000","last_clean":"2024-04-25T00:45:35.599183+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:45:35.599183+0000","last_undegraded":"2024-04-25T00:45:35.599183+0000","last_fullsized":"2024-04-25T00:45:35.599183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491953+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:45:51.491953+0000","last_peered":"2024-04-25T00:45:51.491953+0000","last_clean":"2024-04-25T00:45:51.491953+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:45:51.491953+0000","last_undegraded":"2024-04-25T00:45:51.491953+0000","last_fullsized":"2024-04-25T00:45:51.491953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:45.167958+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:45:45.167958+0000","last_peered":"2024-04-25T00:45:45.167958+0000","last_clean":"2024-04-25T00:45:45.167958+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:45:45.167958+0000","last_undegraded":"2024-04-25T00:45:45.167958+0000","last_fullsized":"2024-04-25T00:45:45.167958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510268+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:45:51.510268+0000","last_peered":"2024-04-25T00:45:51.510268+0000","last_clean":"2024-04-25T00:45:51.510268+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:45:51.510268+0000","last_undegraded":"2024-04-25T00:45:51.510268+0000","last_fullsized":"2024-04-25T00:45:51.510268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'31","reported_seq":50,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.592850+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:45:35.592850+0000","last_peered":"2024-04-25T00:45:35.592850+0000","last_clean":"2024-04-25T00:45:35.592850+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:45:35.592850+0000","last_undegraded":"2024-04-25T00:45:35.592850+0000","last_fullsized":"2024-04-25T00:45:35.592850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521036+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:45:51.521036+0000","last_peered":"2024-04-25T00:45:51.521036+0000","last_clean":"2024-04-25T00:45:51.521036+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:45:51.521036+0000","last_undegraded":"2024-04-25T00:45:51.521036+0000","last_fullsized":"2024-04-25T00:45:51.521036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'508","reported_seq":527,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.577934+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:45:35.577934+0000","last_peered":"2024-04-25T00:45:35.577934+0000","last_clean":"2024-04-25T00:45:35.577934+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:45:35.577934+0000","last_undegraded":"2024-04-25T00:45:35.577934+0000","last_fullsized":"2024-04-25T00:45:35.577934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.508442+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:45:51.508442+0000","last_peered":"2024-04-25T00:45:51.508442+0000","last_clean":"2024-04-25T00:45:51.508442+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:45:51.508442+0000","last_undegraded":"2024-04-25T00:45:51.508442+0000","last_fullsized":"2024-04-25T00:45:51.508442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.535096+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:45:51.535096+0000","last_peered":"2024-04-25T00:45:51.535096+0000","last_clean":"2024-04-25T00:45:51.535096+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:45:51.535096+0000","last_undegraded":"2024-04-25T00:45:51.535096+0000","last_fullsized":"2024-04-25T00:45:51.535096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'16","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.973394+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:45:32.973394+0000","last_peered":"2024-04-25T00:45:32.973394+0000","last_clean":"2024-04-25T00:45:32.973394+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:45:32.973394+0000","last_undegraded":"2024-04-25T00:45:32.973394+0000","last_fullsized":"2024-04-25T00:45:32.973394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533744+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:45:51.533744+0000","last_peered":"2024-04-25T00:45:51.533744+0000","last_clean":"2024-04-25T00:45:51.533744+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:45:51.533744+0000","last_undegraded":"2024-04-25T00:45:51.533744+0000","last_fullsized":"2024-04-25T00:45:51.533744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'62","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:03.994450+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:46:03.994450+0000","last_peered":"2024-04-25T00:46:03.994450+0000","last_clean":"2024-04-25T00:46:03.994450+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:46:03.994450+0000","last_undegraded":"2024-04-25T00:46:03.994450+0000","last_fullsized":"2024-04-25T00:46:03.994450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5442977,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":5338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'57","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509543+0000","last_change":"2024-04-25T00:43:39.829056+0000","last_active":"2024-04-25T00:45:51.509543+0000","last_peered":"2024-04-25T00:45:51.509543+0000","last_clean":"2024-04-25T00:45:51.509543+0000","last_became_active":"2024-04-25T00:43:39.828089+0000","last_became_peered":"2024-04-25T00:43:39.828089+0000","last_unstale":"2024-04-25T00:45:51.509543+0000","last_undegraded":"2024-04-25T00:45:51.509543+0000","last_fullsized":"2024-04-25T00:45:51.509543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:30.063490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'34","reported_seq":53,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.708112+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:45:39.708112+0000","last_peered":"2024-04-25T00:45:39.708112+0000","last_clean":"2024-04-25T00:45:39.708112+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:45:39.708112+0000","last_undegraded":"2024-04-25T00:45:39.708112+0000","last_fullsized":"2024-04-25T00:45:39.708112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510036+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:45:51.510036+0000","last_peered":"2024-04-25T00:45:51.510036+0000","last_clean":"2024-04-25T00:45:51.510036+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:45:51.510036+0000","last_undegraded":"2024-04-25T00:45:51.510036+0000","last_fullsized":"2024-04-25T00:45:51.510036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'48","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.581829+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:45:35.581829+0000","last_peered":"2024-04-25T00:45:35.581829+0000","last_clean":"2024-04-25T00:45:35.581829+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:45:35.581829+0000","last_undegraded":"2024-04-25T00:45:35.581829+0000","last_fullsized":"2024-04-25T00:45:35.581829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":4111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3307,"ondisk_log_size":3307,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":30967508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":3252,"num_write_kb":31581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61997056,"data_stored":61935016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3892,"ondisk_log_size":3892,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57908,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94313932,"statfs":{"total":96636764160,"available":96577466368,"internally_reserved":0,"allocated":18853888,"data_stored":18700145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14069760,"data_stored":13915563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":83396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39604,"kb_used_data":8428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332236,"statfs":{"total":96636764160,"available":96596209664,"internally_reserved":0,"allocated":8630272,"data_stored":8472040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52368,"kb_used_data":12872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319472,"statfs":{"total":96636764160,"available":96583139328,"internally_reserved":0,"allocated":13180928,"data_stored":13024127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":589824,"data_stored":440543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":83466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":9224192,"data_stored":9062920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":357237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12951552,"data_stored":12940821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8388734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13840384,"data_stored":13832257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18030592,"data_stored":18026471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:46:15.105 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:46:15.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 Processor -- start 2024-04-25T00:46:15.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 -- start start 2024-04-25T00:46:15.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f087415b200 0x7f0874072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f087415f980 0x7f08740b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f08740b3f00 0x7f08740b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f08740733a0 con 0x7f08740b3f00 2024-04-25T00:46:15.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f087415b5e0 con 0x7f087415b200 2024-04-25T00:46:15.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0879c20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f08740717e0 con 0x7f087415f980 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f08740b3f00 0x7f08740b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f08740b3f00 0x7f08740b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38164/0 (socket says 172.21.15.129:38164) 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 -- 172.21.15.129:0/1368935535 learned_addr learned my addr 172.21.15.129:0/1368935535 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f08737fe640 1 --2- 172.21.15.129:0/1368935535 >> v2:172.21.15.196:3300/0 conn(0x7f087415b200 0x7f0874072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f086bfff640 1 --2- 172.21.15.129:0/1368935535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f087415f980 0x7f08740b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 -- 172.21.15.129:0/1368935535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f087415f980 msgr2=0x7f08740b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 --2- 172.21.15.129:0/1368935535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f087415f980 0x7f08740b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 -- 172.21.15.129:0/1368935535 >> v2:172.21.15.196:3300/0 conn(0x7f087415b200 msgr2=0x7f0874072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 --2- 172.21.15.129:0/1368935535 >> v2:172.21.15.196:3300/0 conn(0x7f087415b200 0x7f0874072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.259+0000 7f0873fff640 1 -- 172.21.15.129:0/1368935535 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0874071ac0 con 0x7f08740b3f00 2024-04-25T00:46:15.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 --2- 172.21.15.129:0/1368935535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08740b3f00 0x7f08740b0dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f08740678d0 tx=0x7f086c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a176848252138663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:15.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0872ffd640 1 -- 172.21.15.129:0/1368935535 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0a64b0 con 0x7f08740b3f00 2024-04-25T00:46:15.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0872ffd640 1 -- 172.21.15.129:0/1368935535 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f086c0a6650 con 0x7f08740b3f00 2024-04-25T00:46:15.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0872ffd640 1 -- 172.21.15.129:0/1368935535 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0a6a20 con 0x7f08740b3f00 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1368935535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08740b3f00 msgr2=0x7f08740b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- 172.21.15.129:0/1368935535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08740b3f00 0x7f08740b0dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f08740678d0 tx=0x7f086c01c730 comp rx=0 tx=0).stop 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1368935535 shutdown_connections 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- 172.21.15.129:0/1368935535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f087415f980 0x7f08740b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- 172.21.15.129:0/1368935535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08740b3f00 0x7f08740b0dd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- 172.21.15.129:0/1368935535 >> v2:172.21.15.196:3300/0 conn(0x7f087415b200 0x7f0874072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1368935535 >> 172.21.15.129:0/1368935535 conn(0x7f08740ce400 msgr2=0x7f08740b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:15.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1368935535 shutdown_connections 2024-04-25T00:46:15.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1368935535 wait complete. 2024-04-25T00:46:15.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 Processor -- start 2024-04-25T00:46:15.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- start start 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08740b3f00 0x7f0874151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f087415b200 0x7f08741521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 0x7f087415d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0874072880 con 0x7f087415f980 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08740714f0 con 0x7f087415b200 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0874072a00 con 0x7f08740b3f00 2024-04-25T00:46:15.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 0x7f087415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 0x7f087415d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38170/0 (socket says 172.21.15.129:38170) 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 -- 172.21.15.129:0/1824561198 learned_addr learned my addr 172.21.15.129:0/1824561198 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f086bfff640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f087415b200 0x7f08741521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f08737fe640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08740b3f00 0x7f0874151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 -- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08740b3f00 msgr2=0x7f0874151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08740b3f00 0x7f0874151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 -- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f087415b200 msgr2=0x7f08741521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f087415b200 0x7f08741521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 -- 172.21.15.129:0/1824561198 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087414a020 con 0x7f087415f980 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0873fff640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 0x7f087415d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f086c01ade0 tx=0x7f086c0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0bf070 con 0x7f087415f980 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f086c0090d0 con 0x7f087415f980 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087415dea0 con 0x7f087415f980 2024-04-25T00:46:15.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c0a6a40 con 0x7f087415f980 2024-04-25T00:46:15.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.263+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087415e460 con 0x7f087415f980 2024-04-25T00:46:15.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.267+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0874062e50 con 0x7f087415f980 2024-04-25T00:46:15.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.267+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f086c0a6be0 con 0x7f087415f980 2024-04-25T00:46:15.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.267+0000 7f0870ff9640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f08540678c0 0x7f0854069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:15.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.267+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f086c11fb10 con 0x7f087415f980 2024-04-25T00:46:15.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.267+0000 7f08737fe640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f08540678c0 0x7f0854069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:15.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.267+0000 7f08737fe640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f08540678c0 0x7f0854069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f086400ba30 tx=0x7f0864008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:15.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.271+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f086c0165e0 con 0x7f087415f980 2024-04-25T00:46:15.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.419+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08740d0890 con 0x7f087415f980 2024-04-25T00:46:15.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0870ff9640 1 -- 172.21.15.129:0/1824561198 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f086c0f0520 con 0x7f087415f980 2024-04-25T00:46:15.425 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:15.425 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f08540678c0 msgr2=0x7f0854069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f08540678c0 0x7f0854069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f086400ba30 tx=0x7f0864008040 comp rx=0 tx=0).stop 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 msgr2=0x7f087415d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 0x7f087415d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f086c01ade0 tx=0x7f086c0194d0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 shutdown_connections 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f08540678c0 0x7f0854069d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f087415f980 0x7f087415d150 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f087415b200 0x7f08741521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 --2- 172.21.15.129:0/1824561198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08740b3f00 0x7f0874151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:15.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 >> 172.21.15.129:0/1824561198 conn(0x7f08740ce400 msgr2=0x7f087415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:15.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 shutdown_connections 2024-04-25T00:46:15.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:15.423+0000 7f0879c20640 1 -- 172.21.15.129:0/1824561198 wait complete. 2024-04-25T00:46:16.443 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T00:46:16.443 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:16.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.579+0000 7f2b803d0640 1 Processor -- start 2024-04-25T00:46:16.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.579+0000 7f2b803d0640 1 -- start start 2024-04-25T00:46:16.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.579+0000 7f2b803d0640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2b780a9b90 0x7f2b78145700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.579+0000 7f2b803d0640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b780d1500 con 0x7f2b780b0750 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.579+0000 7f2b803d0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b780d17e0 con 0x7f2b78145cd0 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.579+0000 7f2b803d0640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f2b780d2080 con 0x7f2b780a9b90 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2b780a9b90 0x7f2b78145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2b780a9b90 0x7f2b78145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38172/0 (socket says 172.21.15.129:38172) 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 -- 172.21.15.129:0/3054521244 learned_addr learned my addr 172.21.15.129:0/3054521244 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 -- 172.21.15.129:0/3054521244 >> v1:172.21.15.196:6789/0 conn(0x7f2b78145cd0 legacy=0x7f2b7815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 -- 172.21.15.129:0/3054521244 >> v1:172.21.15.129:6789/0 conn(0x7f2b780b0750 legacy=0x7f2b780a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d16a640 1 -- 172.21.15.129:0/3054521244 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054207372 0 0) 0x7f2b780d1500 con 0x7f2b780b0750 2024-04-25T00:46:16.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 -- 172.21.15.129:0/3054521244 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780d1ac0 con 0x7f2b780a9b90 2024-04-25T00:46:16.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d96b640 1 --2- 172.21.15.129:0/3054521244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780a9b90 0x7f2b78145700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009870 tx=0x7f2b6801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824c43b0b0db421 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d16a640 1 -- 172.21.15.129:0/3054521244 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680b0070 con 0x7f2b780a9b90 2024-04-25T00:46:16.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d16a640 1 -- 172.21.15.129:0/3054521244 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b680ab600 con 0x7f2b780a9b90 2024-04-25T00:46:16.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b7d16a640 1 -- 172.21.15.129:0/3054521244 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680ab9d0 con 0x7f2b780a9b90 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 -- 172.21.15.129:0/3054521244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780a9b90 msgr2=0x7f2b78145700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 --2- 172.21.15.129:0/3054521244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780a9b90 0x7f2b78145700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009870 tx=0x7f2b6801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 -- 172.21.15.129:0/3054521244 shutdown_connections 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 --2- 172.21.15.129:0/3054521244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780a9b90 0x7f2b78145700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 -- 172.21.15.129:0/3054521244 >> 172.21.15.129:0/3054521244 conn(0x7f2b7806ee80 msgr2=0x7f2b7806f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 -- 172.21.15.129:0/3054521244 shutdown_connections 2024-04-25T00:46:16.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.583+0000 7f2b803d0640 1 -- 172.21.15.129:0/3054521244 wait complete. 2024-04-25T00:46:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 Processor -- start 2024-04-25T00:46:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- start start 2024-04-25T00:46:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b780a9b90 0x7f2b781586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780b0750 0x7f2b7813df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b78145cd0 0x7f2b78144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78169f80 con 0x7f2b780b0750 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7815df80 con 0x7f2b780a9b90 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7816a100 con 0x7f2b78145cd0 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780b0750 0x7f2b7813df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780b0750 0x7f2b7813df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38176/0 (socket says 172.21.15.129:38176) 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 -- 172.21.15.129:0/2556711936 learned_addr learned my addr 172.21.15.129:0/2556711936 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 -- 172.21.15.129:0/2556711936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b78145cd0 msgr2=0x7f2b78144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7e96d640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b78145cd0 0x7f2b78144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b78145cd0 0x7f2b78144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7e16c640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b780a9b90 0x7f2b781586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 -- 172.21.15.129:0/2556711936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b780a9b90 msgr2=0x7f2b781586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b780a9b90 0x7f2b781586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 -- 172.21.15.129:0/2556711936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b78143a50 con 0x7f2b780b0750 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b7d96b640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780b0750 0x7f2b7813df60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2b680ae040 tx=0x7f2b6801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680b0040 con 0x7f2b780b0750 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68004030 con 0x7f2b780b0750 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b78143d30 con 0x7f2b780b0750 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004400 con 0x7f2b780b0750 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7813e4a0 con 0x7f2b780b0750 2024-04-25T00:46:16.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.587+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b78158dd0 con 0x7f2b780b0750 2024-04-25T00:46:16.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b68003620 con 0x7f2b780b0750 2024-04-25T00:46:16.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b5c0678c0 0x7f2b5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f2b68120b20 con 0x7f2b780b0750 2024-04-25T00:46:16.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b7e16c640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b5c0678c0 0x7f2b5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7f2b5c06fcc0 0x7f2b5c072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 --> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b680ab600 con 0x7f2b5c06fcc0 2024-04-25T00:46:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2b68120fb0 con 0x7f2b780b0750 2024-04-25T00:46:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b7e96d640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7f2b5c06fcc0 0x7f2b5c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b7e16c640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b5c0678c0 0x7f2b5c069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b74007fd0 tx=0x7f2b74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b7e96d640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7f2b5c06fcc0 0x7f2b5c072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.591+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== osd.0 v2:172.21.15.129:6800/1911044788 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b680ab600 con 0x7f2b5c06fcc0 2024-04-25T00:46:16.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 --> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2b78062e50 con 0x7f2b5c06fcc0 2024-04-25T00:46:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b6effd640 1 -- 172.21.15.129:0/2556711936 <== osd.0 v2:172.21.15.129:6800/1911044788 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2b78062e50 con 0x7f2b5c06fcc0 2024-04-25T00:46:16.607 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:16.607 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:16.607 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:16.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7f2b5c06fcc0 msgr2=0x7f2b5c072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6800/1911044788,v1:172.21.15.129:6803/1911044788] conn(0x7f2b5c06fcc0 0x7f2b5c072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b5c0678c0 msgr2=0x7f2b5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2b5c0678c0 0x7f2b5c069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b74007fd0 tx=0x7f2b74009040 comp rx=0 tx=0).stop 2024-04-25T00:46:16.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780b0750 msgr2=0x7f2b7813df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.603+0000 7f2b803d0640 1 --2- 172.21.15.129:0/2556711936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b780b0750 0x7f2b7813df60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2b680ae040 tx=0x7f2b6801ad10 comp rx=0 tx=0).stop 2024-04-25T00:46:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.607+0000 7f2b7e96d640 1 -- 172.21.15.129:0/2556711936 reap_dead start 2024-04-25T00:46:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.607+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 shutdown_connections 2024-04-25T00:46:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.607+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 >> 172.21.15.129:0/2556711936 conn(0x7f2b7806ee80 msgr2=0x7f2b780ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.607+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 shutdown_connections 2024-04-25T00:46:16.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.607+0000 7f2b803d0640 1 -- 172.21.15.129:0/2556711936 wait complete. 2024-04-25T00:46:16.623 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T00:46:16.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb14dedc640 1 Processor -- start 2024-04-25T00:46:16.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb14dedc640 1 -- start start 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb14dedc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb148141530 0x7fb148067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb14dedc640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb14815f940 con 0x7fb148072e80 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb14dedc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb148141910 con 0x7fb1480cff90 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb14dedc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb1480cf7b0 con 0x7fb148141530 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fb148072e80 0x7fb14815f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42070/0 (socket says 172.21.15.129:42070) 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 -- 172.21.15.129:0/3790869103 learned_addr learned my addr 172.21.15.129:0/3790869103 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3062786721 0 0) 0x7fb14815f940 con 0x7fb148072e80 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1477fe640 1 --2- 172.21.15.129:0/3790869103 >> v2:172.21.15.196:3300/0 conn(0x7fb148141530 0x7fb148067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb134003530 con 0x7fb148072e80 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638992536 0 0) 0x7fb134003530 con 0x7fb148072e80 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 >> v2:172.21.15.196:3300/0 conn(0x7fb148141530 msgr2=0x7fb148067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 --2- 172.21.15.129:0/3790869103 >> v2:172.21.15.196:3300/0 conn(0x7fb148141530 0x7fb148067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 >> v1:172.21.15.196:6789/0 conn(0x7fb1480cff90 legacy=0x7fb14815eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1480ce730 con 0x7fb148072e80 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fb138001a00 con 0x7fb148072e80 2024-04-25T00:46:16.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 >> v1:172.21.15.129:6789/0 conn(0x7fb148072e80 legacy=0x7fb14815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1340039c0 0x7fb134003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb134004810 0x7fb134004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1340053d0 0x7fb13400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb13400c230 con 0x7fb1340053d0 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb13400c3e0 con 0x7fb1340039c0 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb13400c5e0 con 0x7fb134004810 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1340053d0 0x7fb13400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1477fe640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1340039c0 0x7fb134003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb146ffd640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb134004810 0x7fb134004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 -- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb134004810 msgr2=0x7fb134004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb134004810 0x7fb134004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 -- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1340039c0 msgr2=0x7fb134003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1340039c0 0x7fb134003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb147fff640 1 -- 172.21.15.129:0/3790869103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb134003530 con 0x7fb1340053d0 2024-04-25T00:46:16.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.731+0000 7fb1477fe640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1340039c0 0x7fb134003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb147fff640 1 -- 172.21.15.129:0/3790869103 reap_dead start 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb147fff640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1340053d0 0x7fb13400b9d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb13800a860 tx=0x7fb13801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17db69302c7b8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800a480 con 0x7fb1340053d0 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1380a5e50 con 0x7fb1340053d0 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb1467fc640 1 -- 172.21.15.129:0/3790869103 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1380ad850 con 0x7fb1340053d0 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- 172.21.15.129:0/3790869103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1340053d0 msgr2=0x7fb13400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1340053d0 0x7fb13400b9d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb13800a860 tx=0x7fb13801d380 comp rx=0 tx=0).stop 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- 172.21.15.129:0/3790869103 shutdown_connections 2024-04-25T00:46:16.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3790869103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1340053d0 0x7fb13400b9d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- 172.21.15.129:0/3790869103 >> 172.21.15.129:0/3790869103 conn(0x7fb1480b1c10 msgr2=0x7fb1480b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- 172.21.15.129:0/3790869103 shutdown_connections 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- 172.21.15.129:0/3790869103 wait complete. 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 Processor -- start 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- start start 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1340053d0 0x7fb148134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 0x7fb14815b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb14815b980 0x7fb14813ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb148144a60 con 0x7fb14815aa80 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1480679c0 con 0x7fb14815b980 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb14dedc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb148067070 con 0x7fb1340053d0 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb146ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 0x7fb14815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb146ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 0x7fb14815b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38194/0 (socket says 172.21.15.129:38194) 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.735+0000 7fb146ffd640 1 -- 172.21.15.129:0/3660806009 learned_addr learned my addr 172.21.15.129:0/3660806009 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb1477fe640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1340053d0 0x7fb148134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb146ffd640 1 -- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1340053d0 msgr2=0x7fb148134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb146ffd640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1340053d0 0x7fb148134380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb147fff640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb14815b980 0x7fb14813ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb146ffd640 1 -- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb14815b980 msgr2=0x7fb14813ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb146ffd640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb14815b980 0x7fb14813ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb146ffd640 1 -- 172.21.15.129:0/3660806009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14813eed0 con 0x7fb14815aa80 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb146ffd640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 0x7fb14815b440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb13000d4c0 tx=0x7fb13000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130005480 con 0x7fb14815aa80 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb130017070 con 0x7fb14815aa80 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14813f1b0 con 0x7fb14815aa80 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130003840 con 0x7fb14815aa80 2024-04-25T00:46:16.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14813f770 con 0x7fb14815aa80 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb148062e50 con 0x7fb14815aa80 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb1300039e0 con 0x7fb14815aa80 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb13ffff640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb1280678f0 0x7fb128069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb130085b10 con 0x7fb14815aa80 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb1477fe640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb1280678f0 0x7fb128069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.739+0000 7fb1477fe640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb1280678f0 0x7fb128069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c890 tx=0x7fb140008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:16.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.743+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb13000f2f0 con 0x7fb14815aa80 2024-04-25T00:46:16.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.891+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb14815bd60 con 0x7fb1280678f0 2024-04-25T00:46:16.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.891+0000 7fb13ffff640 1 -- 172.21.15.129:0/3660806009 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb14815bd60 con 0x7fb1280678f0 2024-04-25T00:46:16.894 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb1280678f0 msgr2=0x7fb128069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb1280678f0 0x7fb128069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c890 tx=0x7fb140008040 comp rx=0 tx=0).stop 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 msgr2=0x7fb14815b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 0x7fb14815b440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb13000d4c0 tx=0x7fb13000d980 comp rx=0 tx=0).stop 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 shutdown_connections 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb1280678f0 0x7fb128069db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb14815b980 0x7fb14813ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb14815aa80 0x7fb14815b440 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 --2- 172.21.15.129:0/3660806009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1340053d0 0x7fb148134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:16.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 >> 172.21.15.129:0/3660806009 conn(0x7fb1480b1c10 msgr2=0x7fb14813a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:16.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 shutdown_connections 2024-04-25T00:46:16.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:16.895+0000 7fb14dedc640 1 -- 172.21.15.129:0/3660806009 wait complete. 2024-04-25T00:46:16.913 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T00:46:16.913 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:17.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 Processor -- start 2024-04-25T00:46:17.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 -- start start 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc1553d0 0x7f01bc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f01bc15c150 0x7f01bc0cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc0cdeb0 con 0x7f01bc0cf8a0 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f01bc0cd2a0 con 0x7f01bc15c150 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01c1954640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc0cd580 con 0x7f01bc1553d0 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01bb7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f01bc0cf8a0 0x7f01bc15e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42082/0 (socket says 172.21.15.129:42082) 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01bb7fe640 1 -- 172.21.15.129:0/3151386373 learned_addr learned my addr 172.21.15.129:0/3151386373 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.035+0000 7f01b27fc640 1 --2- 172.21.15.129:0/3151386373 >> v2:172.21.15.129:3300/0 conn(0x7f01bc15c150 0x7f01bc0cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01baffd640 1 --2- 172.21.15.129:0/3151386373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc1553d0 0x7f01bc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01ba7fc640 1 -- 172.21.15.129:0/3151386373 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974656370 0 0) 0x7f01bc0cdeb0 con 0x7f01bc0cf8a0 2024-04-25T00:46:17.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01ba7fc640 1 -- 172.21.15.129:0/3151386373 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01a8003530 con 0x7f01bc0cf8a0 2024-04-25T00:46:17.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01b27fc640 1 -- 172.21.15.129:0/3151386373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc1553d0 msgr2=0x7f01bc0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01b27fc640 1 --2- 172.21.15.129:0/3151386373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc1553d0 0x7f01bc0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01b27fc640 1 -- 172.21.15.129:0/3151386373 >> v1:172.21.15.129:6789/0 conn(0x7f01bc0cf8a0 legacy=0x7f01bc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01b27fc640 1 -- 172.21.15.129:0/3151386373 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc0cd860 con 0x7f01bc15c150 2024-04-25T00:46:17.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01b27fc640 1 --2- 172.21.15.129:0/3151386373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc0cf360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a4020840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a7abeefb756316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01ba7fc640 1 -- 172.21.15.129:0/3151386373 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40b6070 con 0x7f01bc15c150 2024-04-25T00:46:17.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01ba7fc640 1 -- 172.21.15.129:0/3151386373 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a40a9750 con 0x7f01bc15c150 2024-04-25T00:46:17.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01ba7fc640 1 -- 172.21.15.129:0/3151386373 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40a9b20 con 0x7f01bc15c150 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 -- 172.21.15.129:0/3151386373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 msgr2=0x7f01bc0cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 --2- 172.21.15.129:0/3151386373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc0cf360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a4020840 comp rx=0 tx=0).stop 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 -- 172.21.15.129:0/3151386373 shutdown_connections 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 --2- 172.21.15.129:0/3151386373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc0cf360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 --2- 172.21.15.129:0/3151386373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc1553d0 0x7f01bc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 -- 172.21.15.129:0/3151386373 >> 172.21.15.129:0/3151386373 conn(0x7f01bc067760 msgr2=0x7f01bc0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 -- 172.21.15.129:0/3151386373 shutdown_connections 2024-04-25T00:46:17.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.039+0000 7f01c1954640 1 -- 172.21.15.129:0/3151386373 wait complete. 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 Processor -- start 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- start start 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc0cf8a0 0x7f01bc14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01bc1553d0 0x7f01bc06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0ccff0 con 0x7f01bc15c150 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc0cce70 con 0x7f01bc1553d0 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc070f20 con 0x7f01bc0cf8a0 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38202/0 (socket says 172.21.15.129:38202) 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 -- 172.21.15.129:0/1426870376 learned_addr learned my addr 172.21.15.129:0/1426870376 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 -- 172.21.15.129:0/1426870376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc0cf8a0 msgr2=0x7f01bc14abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01bc0cf8a0 0x7f01bc14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 -- 172.21.15.129:0/1426870376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01bc1553d0 msgr2=0x7f01bc06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01bc1553d0 0x7f01bc06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 -- 172.21.15.129:0/1426870376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc156700 con 0x7f01bc15c150 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01bb7fe640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc06e8f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c7a0 tx=0x7f01b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4018070 con 0x7f01bc15c150 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b400a640 con 0x7f01bc15c150 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b400a9f0 con 0x7f01bc15c150 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc1569e0 con 0x7f01bc15c150 2024-04-25T00:46:17.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc156fa0 con 0x7f01bc15c150 2024-04-25T00:46:17.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01bc1564b0 con 0x7f01bc15c150 2024-04-25T00:46:17.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.043+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01b401b070 con 0x7f01bc15c150 2024-04-25T00:46:17.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b8ff9640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f019c0678c0 0x7f019c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01baffd640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f019c0678c0 0x7f019c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f01b4085c50 con 0x7f01bc15c150 2024-04-25T00:46:17.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b8ff9640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f019c06fcc0 0x7f019c072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 --> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01b401be50 con 0x7f019c06fcc0 2024-04-25T00:46:17.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f01b40860e0 con 0x7f01bc15c150 2024-04-25T00:46:17.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b27fc640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f019c06fcc0 0x7f019c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01baffd640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f019c0678c0 0x7f019c069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098f0 tx=0x7f01ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b27fc640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f019c06fcc0 0x7f019c072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.047+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== osd.1 v2:172.21.15.129:6801/1082494394 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01b401be50 con 0x7f019c06fcc0 2024-04-25T00:46:17.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 --> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f01bc062e50 con 0x7f019c06fcc0 2024-04-25T00:46:17.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01b8ff9640 1 -- 172.21.15.129:0/1426870376 <== osd.1 v2:172.21.15.129:6801/1082494394 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f01bc062e50 con 0x7f019c06fcc0 2024-04-25T00:46:17.062 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:17.062 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:17.063 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f019c06fcc0 msgr2=0x7f019c072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6801/1082494394,v1:172.21.15.129:6802/1082494394] conn(0x7f019c06fcc0 0x7f019c072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f019c0678c0 msgr2=0x7f019c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f019c0678c0 0x7f019c069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098f0 tx=0x7f01ac008040 comp rx=0 tx=0).stop 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 msgr2=0x7f01bc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 --2- 172.21.15.129:0/1426870376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01bc15c150 0x7f01bc06e8f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c7a0 tx=0x7f01b400cc60 comp rx=0 tx=0).stop 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01bb7fe640 1 -- 172.21.15.129:0/1426870376 reap_dead start 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 shutdown_connections 2024-04-25T00:46:17.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 >> 172.21.15.129:0/1426870376 conn(0x7f01bc067760 msgr2=0x7f01bc0af940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 shutdown_connections 2024-04-25T00:46:17.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.059+0000 7f01c1954640 1 -- 172.21.15.129:0/1426870376 wait complete. 2024-04-25T00:46:17.079 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T00:46:17.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0438b48640 1 Processor -- start 2024-04-25T00:46:17.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0438b48640 1 -- start start 2024-04-25T00:46:17.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0438b48640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f04340d0bb0 0x7f043415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0438b48640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0434161e30 con 0x7f04341479a0 2024-04-25T00:46:17.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0438b48640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04340a96b0 con 0x7f043406b420 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0438b48640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f04340a9480 con 0x7f04340d0bb0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043259c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f04341479a0 0x7f04341625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42090/0 (socket says 172.21.15.129:42090) 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043259c640 1 -- 172.21.15.129:0/1452778584 learned_addr learned my addr 172.21.15.129:0/1452778584 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 --2- 172.21.15.129:0/1452778584 >> v2:172.21.15.196:3300/0 conn(0x7f04340d0bb0 0x7f043415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2192038751 0 0) 0x7f0434161e30 con 0x7f04341479a0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f041c003530 con 0x7f04341479a0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2075128834 0 0) 0x7f041c003530 con 0x7f04341479a0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 >> v2:172.21.15.196:3300/0 conn(0x7f04340d0bb0 msgr2=0x7f043415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 --2- 172.21.15.129:0/1452778584 >> v2:172.21.15.196:3300/0 conn(0x7f04340d0bb0 0x7f043415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 >> v1:172.21.15.196:6789/0 conn(0x7f043406b420 legacy=0x7f04340cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04340a9990 con 0x7f04341479a0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f0424001a00 con 0x7f04341479a0 2024-04-25T00:46:17.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 >> v1:172.21.15.129:6789/0 conn(0x7f04341479a0 legacy=0x7f04341625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f041c0039c0 0x7f041c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f041c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f041c0051d0 0x7f041c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f041c00c140 con 0x7f041c004810 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f041c00c2f0 con 0x7f041c0039c0 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f041c00c4f0 con 0x7f041c0051d0 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f041c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043259c640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f041c0039c0 0x7f041c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 -- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f041c0051d0 msgr2=0x7f041c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0432d9d640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f041c0051d0 0x7f041c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f041c0051d0 0x7f041c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 -- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f041c0039c0 msgr2=0x7f041c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f041c0039c0 0x7f041c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0431d9b640 1 -- 172.21.15.129:0/1452778584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c003530 con 0x7f041c004810 2024-04-25T00:46:17.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0432d9d640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f041c0051d0 0x7f041c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f043259c640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f041c0039c0 0x7f041c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.195+0000 7f0432d9d640 1 -- 172.21.15.129:0/1452778584 reap_dead start 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0431d9b640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f041c004c90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0428009ab0 tx=0x7f042801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1317f096c50925e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04280096b0 con 0x7f041c004810 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04280a6430 con 0x7f041c004810 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f043159a640 1 -- 172.21.15.129:0/1452778584 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04280a6820 con 0x7f041c004810 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 -- 172.21.15.129:0/1452778584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 msgr2=0x7f041c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f041c004c90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0428009ab0 tx=0x7f042801ca50 comp rx=0 tx=0).stop 2024-04-25T00:46:17.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 -- 172.21.15.129:0/1452778584 shutdown_connections 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 --2- 172.21.15.129:0/1452778584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f041c004c90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 -- 172.21.15.129:0/1452778584 >> 172.21.15.129:0/1452778584 conn(0x7f04340b2380 msgr2=0x7f04340b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 -- 172.21.15.129:0/1452778584 shutdown_connections 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 -- 172.21.15.129:0/1452778584 wait complete. 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 Processor -- start 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.199+0000 7f0438b48640 1 -- start start 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f043413d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04341fba90 0x7f0434072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f043415f960 0x7f043415fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04340cf0e0 con 0x7f041c004810 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04340cef60 con 0x7f04341fba90 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04340cf260 con 0x7f043415f960 2024-04-25T00:46:17.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f043413d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f043413d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38220/0 (socket says 172.21.15.129:38220) 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 -- 172.21.15.129:0/3714514492 learned_addr learned my addr 172.21.15.129:0/3714514492 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 -- 172.21.15.129:0/3714514492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f043415f960 msgr2=0x7f043415fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f043415f960 0x7f043415fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 -- 172.21.15.129:0/3714514492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04341fba90 msgr2=0x7f0434072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04341fba90 0x7f0434072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 -- 172.21.15.129:0/3714514492 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0434070a80 con 0x7f041c004810 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f043259c640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f043413d280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0424001380 tx=0x7f042400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424019070 con 0x7f041c004810 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0424004d70 con 0x7f041c004810 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0434070d60 con 0x7f041c004810 2024-04-25T00:46:17.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240072d0 con 0x7f041c004810 2024-04-25T00:46:17.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0434071320 con 0x7f041c004810 2024-04-25T00:46:17.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0434062e50 con 0x7f041c004810 2024-04-25T00:46:17.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f042401c070 con 0x7f041c004810 2024-04-25T00:46:17.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0422ffd640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f04140678f0 0x7f0414069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f0424085e00 con 0x7f041c004810 2024-04-25T00:46:17.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0431d9b640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f04140678f0 0x7f0414069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.203+0000 7f0431d9b640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f04140678f0 0x7f0414069db0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04280ad000 tx=0x7f04280a9040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.207+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0424056240 con 0x7f041c004810 2024-04-25T00:46:17.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.355+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f0434071b20 con 0x7f04140678f0 2024-04-25T00:46:17.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.355+0000 7f0422ffd640 1 -- 172.21.15.129:0/3714514492 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0434071b20 con 0x7f04140678f0 2024-04-25T00:46:17.359 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T00:46:17.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f04140678f0 msgr2=0x7f0414069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f04140678f0 0x7f0414069db0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04280ad000 tx=0x7f04280a9040 comp rx=0 tx=0).stop 2024-04-25T00:46:17.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 msgr2=0x7f043413d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f043413d280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0424001380 tx=0x7f042400b4a0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 shutdown_connections 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f04140678f0 0x7f0414069db0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f043415f960 0x7f043415fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04341fba90 0x7f0434072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 --2- 172.21.15.129:0/3714514492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041c004810 0x7f043413d280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 >> 172.21.15.129:0/3714514492 conn(0x7f04340b2380 msgr2=0x7f043415dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 shutdown_connections 2024-04-25T00:46:17.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.359+0000 7f0438b48640 1 -- 172.21.15.129:0/3714514492 wait complete. 2024-04-25T00:46:17.377 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T00:46:17.377 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:17.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabbdee2640 1 Processor -- start 2024-04-25T00:46:17.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabbdee2640 1 -- start start 2024-04-25T00:46:17.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabbdee2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fabb815f320 0x7fabb80aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabbdee2640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb80d2e10 con 0x7fabb80acf70 2024-04-25T00:46:17.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabbdee2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabb80d30f0 con 0x7fabb80d27f0 2024-04-25T00:46:17.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabbdee2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fabb815f700 con 0x7fabb815f320 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fabb80acf70 0x7fabb80d20e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42106/0 (socket says 172.21.15.129:42106) 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 -- 172.21.15.129:0/319579114 learned_addr learned my addr 172.21.15.129:0/319579114 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb77fe640 1 --2- 172.21.15.129:0/319579114 >> v2:172.21.15.196:3300/0 conn(0x7fabb815f320 0x7fabb80aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3046683370 0 0) 0x7fabb80d2e10 con 0x7fabb80acf70 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faba0003530 con 0x7fabb80acf70 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1773175931 0 0) 0x7faba0003530 con 0x7fabb80acf70 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 >> v2:172.21.15.196:3300/0 conn(0x7fabb815f320 msgr2=0x7fabb80aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 --2- 172.21.15.129:0/319579114 >> v2:172.21.15.196:3300/0 conn(0x7fabb815f320 0x7fabb80aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 >> v1:172.21.15.196:6789/0 conn(0x7fabb80d27f0 legacy=0x7fabb815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb80d33d0 con 0x7fabb80acf70 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fabac001680 con 0x7fabb80acf70 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 >> v1:172.21.15.129:6789/0 conn(0x7fabb80acf70 legacy=0x7fabb80d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba00039c0 0x7faba0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faba0004810 0x7faba0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba00051d0 0x7faba000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabb815f700 con 0x7faba0004810 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faba000c170 con 0x7faba00051d0 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faba000c370 con 0x7faba00039c0 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faba0004810 0x7faba0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb77fe640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba00039c0 0x7faba0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb7fff640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba00051d0 0x7faba000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 -- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba00039c0 msgr2=0x7faba0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba00039c0 0x7faba0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 -- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba00051d0 msgr2=0x7faba000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba00051d0 0x7faba000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb6ffd640 1 -- 172.21.15.129:0/319579114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba0003530 con 0x7faba0004810 2024-04-25T00:46:17.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb7fff640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faba00051d0 0x7faba000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.503+0000 7fabb7fff640 1 -- 172.21.15.129:0/319579114 reap_dead start 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faba0004810 0x7faba0004c90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c240 tx=0x7fabac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4741713b15dd3bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac007d20 con 0x7faba0004810 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabac0a9070 con 0x7faba0004810 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabb67fc640 1 -- 172.21.15.129:0/319579114 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac005d80 con 0x7faba0004810 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 -- 172.21.15.129:0/319579114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faba0004810 msgr2=0x7faba0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faba0004810 0x7faba0004c90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c240 tx=0x7fabac01cc80 comp rx=0 tx=0).stop 2024-04-25T00:46:17.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 -- 172.21.15.129:0/319579114 shutdown_connections 2024-04-25T00:46:17.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 --2- 172.21.15.129:0/319579114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faba0004810 0x7faba0004c90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 -- 172.21.15.129:0/319579114 >> 172.21.15.129:0/319579114 conn(0x7fabb8067240 msgr2=0x7fabb80d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 -- 172.21.15.129:0/319579114 shutdown_connections 2024-04-25T00:46:17.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.507+0000 7fabbdee2640 1 -- 172.21.15.129:0/319579114 wait complete. 2024-04-25T00:46:17.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 Processor -- start 2024-04-25T00:46:17.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- start start 2024-04-25T00:46:17.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba0004810 0x7fabb8157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb81eabc0 0x7fabb8140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb812d590 0x7fabb8142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8162870 con 0x7fabb81eabc0 2024-04-25T00:46:17.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81626f0 con 0x7fabb812d590 2024-04-25T00:46:17.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fabb81629f0 con 0x7faba0004810 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb81eabc0 0x7fabb8140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb81eabc0 0x7fabb8140460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38240/0 (socket says 172.21.15.129:38240) 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 -- 172.21.15.129:0/1363382006 learned_addr learned my addr 172.21.15.129:0/1363382006 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb77fe640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba0004810 0x7fabb8157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb7fff640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb812d590 0x7fabb8142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 -- 172.21.15.129:0/1363382006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba0004810 msgr2=0x7fabb8157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faba0004810 0x7fabb8157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 -- 172.21.15.129:0/1363382006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb812d590 msgr2=0x7fabb8142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabb812d590 0x7fabb8142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 -- 172.21.15.129:0/1363382006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb81eb130 con 0x7fabb81eabc0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabb6ffd640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb81eabc0 0x7fabb8140460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fabac01c6f0 tx=0x7faba0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac007d20 con 0x7fabb81eabc0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabac001480 con 0x7fabb81eabc0 2024-04-25T00:46:17.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb81eb410 con 0x7fabb81eabc0 2024-04-25T00:46:17.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac007380 con 0x7fabb81eabc0 2024-04-25T00:46:17.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81eb9d0 con 0x7fabb81eabc0 2024-04-25T00:46:17.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabb812f950 con 0x7fabb81eabc0 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.511+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fabac007520 con 0x7fabb81eabc0 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fab9ffff640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faba40678c0 0x7faba4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fabac11fbe0 con 0x7fabb81eabc0 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fabb77fe640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faba40678c0 0x7faba4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fab9ffff640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7faba406fcc0 0x7faba4072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 --> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabac0bae80 con 0x7faba406fcc0 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fabac120070 con 0x7fabb81eabc0 2024-04-25T00:46:17.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fabb7fff640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7faba406fcc0 0x7faba4072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fabb77fe640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faba40678c0 0x7faba4069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ab0 tx=0x7faba8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fabb7fff640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7faba406fcc0 0x7faba4072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.515+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== osd.2 v2:172.21.15.129:6824/4081708389 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabac0bae80 con 0x7faba406fcc0 2024-04-25T00:46:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 --> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fabb8062e50 con 0x7faba406fcc0 2024-04-25T00:46:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fab9ffff640 1 -- 172.21.15.129:0/1363382006 <== osd.2 v2:172.21.15.129:6824/4081708389 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fabb8062e50 con 0x7faba406fcc0 2024-04-25T00:46:17.530 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:17.530 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:17.530 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7faba406fcc0 msgr2=0x7faba4072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6824/4081708389,v1:172.21.15.129:6825/4081708389] conn(0x7faba406fcc0 0x7faba4072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faba40678c0 msgr2=0x7faba4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7faba40678c0 0x7faba4069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ab0 tx=0x7faba8008040 comp rx=0 tx=0).stop 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb81eabc0 msgr2=0x7fabb8140460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 --2- 172.21.15.129:0/1363382006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb81eabc0 0x7fabb8140460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fabac01c6f0 tx=0x7faba0005840 comp rx=0 tx=0).stop 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabb7fff640 1 -- 172.21.15.129:0/1363382006 reap_dead start 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 shutdown_connections 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 >> 172.21.15.129:0/1363382006 conn(0x7fabb8067240 msgr2=0x7fabb81469b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 shutdown_connections 2024-04-25T00:46:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.527+0000 7fabbdee2640 1 -- 172.21.15.129:0/1363382006 wait complete. 2024-04-25T00:46:17.546 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T00:46:17.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd33956640 1 Processor -- start 2024-04-25T00:46:17.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd33956640 1 -- start start 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd33956640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcd2c144eb0 0x7fcd2c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd33956640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd2c06fc70 con 0x7fcd2c0c9370 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd33956640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd2c162a60 con 0x7fcd2c1483f0 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd33956640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcd2c162d40 con 0x7fcd2c144eb0 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd31ef3640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fcd2c0c9370 0x7fcd2c06f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42108/0 (socket says 172.21.15.129:42108) 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd31ef3640 1 -- 172.21.15.129:0/2536396434 learned_addr learned my addr 172.21.15.129:0/2536396434 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:17.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd316f2640 1 --2- 172.21.15.129:0/2536396434 >> v2:172.21.15.196:3300/0 conn(0x7fcd2c144eb0 0x7fcd2c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13014761 0 0) 0x7fcd2c06fc70 con 0x7fcd2c0c9370 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.659+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd18003530 con 0x7fcd2c0c9370 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641786911 0 0) 0x7fcd18003530 con 0x7fcd2c0c9370 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 >> v2:172.21.15.196:3300/0 conn(0x7fcd2c144eb0 msgr2=0x7fcd2c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 --2- 172.21.15.129:0/2536396434 >> v2:172.21.15.196:3300/0 conn(0x7fcd2c144eb0 0x7fcd2c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 >> v1:172.21.15.196:6789/0 conn(0x7fcd2c1483f0 legacy=0x7fcd2c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c163020 con 0x7fcd2c0c9370 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fcd24001a00 con 0x7fcd2c0c9370 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 >> v1:172.21.15.129:6789/0 conn(0x7fcd2c0c9370 legacy=0x7fcd2c06f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd180039c0 0x7fcd18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd18004810 0x7fcd18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd180051d0 0x7fcd1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c162d40 con 0x7fcd18004810 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800c070 con 0x7fcd180051d0 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1800c270 con 0x7fcd180039c0 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd18004810 0x7fcd18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd31ef3640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd180051d0 0x7fcd1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd316f2640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd180039c0 0x7fcd18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 -- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd180039c0 msgr2=0x7fcd18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd180039c0 0x7fcd18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 -- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd180051d0 msgr2=0x7fcd1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd180051d0 0x7fcd1800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd31ef3640 1 -- 172.21.15.129:0/2536396434 reap_dead start 2024-04-25T00:46:17.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 -- 172.21.15.129:0/2536396434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd18003530 con 0x7fcd18004810 2024-04-25T00:46:17.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd316f2640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd180039c0 0x7fcd18003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd30ef1640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd18004810 0x7fcd18004c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c001780 tx=0x7fcd1c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c657940a262b52a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0013a0 con 0x7fcd18004810 2024-04-25T00:46:17.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0052f0 con 0x7fcd18004810 2024-04-25T00:46:17.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd23fff640 1 -- 172.21.15.129:0/2536396434 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c003840 con 0x7fcd18004810 2024-04-25T00:46:17.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 -- 172.21.15.129:0/2536396434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd18004810 msgr2=0x7fcd18004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd18004810 0x7fcd18004c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c001780 tx=0x7fcd1c01c400 comp rx=0 tx=0).stop 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 -- 172.21.15.129:0/2536396434 shutdown_connections 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 --2- 172.21.15.129:0/2536396434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd18004810 0x7fcd18004c90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 -- 172.21.15.129:0/2536396434 >> 172.21.15.129:0/2536396434 conn(0x7fcd2c06b210 msgr2=0x7fcd2c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 -- 172.21.15.129:0/2536396434 shutdown_connections 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.663+0000 7fcd33956640 1 -- 172.21.15.129:0/2536396434 wait complete. 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 Processor -- start 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- start start 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd18004810 0x7fcd2c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd2c0cdd60 0x7fcd2c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 0x7fcd2c1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c162770 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c1625f0 con 0x7fcd18004810 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2c145290 con 0x7fcd2c0cdd60 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 0x7fcd2c1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 0x7fcd2c1f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38258/0 (socket says 172.21.15.129:38258) 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 -- 172.21.15.129:0/2466232120 learned_addr learned my addr 172.21.15.129:0/2466232120 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd316f2640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd18004810 0x7fcd2c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd30ef1640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd2c0cdd60 0x7fcd2c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 -- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd2c0cdd60 msgr2=0x7fcd2c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd2c0cdd60 0x7fcd2c0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 -- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd18004810 msgr2=0x7fcd2c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd18004810 0x7fcd2c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 -- 172.21.15.129:0/2466232120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c135ec0 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd30ef1640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd2c0cdd60 0x7fcd2c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd31ef3640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 0x7fcd2c1f44b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c162550 tx=0x7fcd240119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24010040 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2401a070 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c136160 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24013a20 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c136720 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c062e50 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.667+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd24013bc0 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.671+0000 7fcd21ffb640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcd100678f0 0x7fcd10069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.671+0000 7fcd316f2640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcd100678f0 0x7fcd10069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.671+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fcd24088c10 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.671+0000 7fcd316f2640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcd100678f0 0x7fcd10069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c067040 tx=0x7fcd28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.675+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcd24059610 con 0x7fcd2c0ca0d0 2024-04-25T00:46:17.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.835+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fcd2c0ce140 con 0x7fcd100678f0 2024-04-25T00:46:17.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd21ffb640 1 -- 172.21.15.129:0/2466232120 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcd28009700 con 0x7fcd100678f0 2024-04-25T00:46:17.839 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcd100678f0 msgr2=0x7fcd10069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcd100678f0 0x7fcd10069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c067040 tx=0x7fcd28008040 comp rx=0 tx=0).stop 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 msgr2=0x7fcd2c1f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 0x7fcd2c1f44b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c162550 tx=0x7fcd240119f0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 shutdown_connections 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fcd100678f0 0x7fcd10069db0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd2c0cdd60 0x7fcd2c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c0ca0d0 0x7fcd2c1f44b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 --2- 172.21.15.129:0/2466232120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd18004810 0x7fcd2c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 >> 172.21.15.129:0/2466232120 conn(0x7fcd2c06b210 msgr2=0x7fcd2c1610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 shutdown_connections 2024-04-25T00:46:17.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.839+0000 7fcd33956640 1 -- 172.21.15.129:0/2466232120 wait complete. 2024-04-25T00:46:17.857 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T00:46:17.858 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:17.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe993a2f640 1 Processor -- start 2024-04-25T00:46:17.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe993a2f640 1 -- start start 2024-04-25T00:46:17.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe993a2f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe98c1608f0 0x7fe98c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe993a2f640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98c0ced50 con 0x7fe98c144fc0 2024-04-25T00:46:17.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe993a2f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98c15fe10 con 0x7fe98c06f390 2024-04-25T00:46:17.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe993a2f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe98c1453a0 con 0x7fe98c1608f0 2024-04-25T00:46:17.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe9917cb640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fe98c144fc0 0x7fe98c0ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42114/0 (socket says 172.21.15.129:42114) 2024-04-25T00:46:17.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe9917cb640 1 -- 172.21.15.129:0/3754771521 learned_addr learned my addr 172.21.15.129:0/3754771521 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097941225 0 0) 0x7fe98c0ced50 con 0x7fe98c144fc0 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe978003530 con 0x7fe98c144fc0 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 --2- 172.21.15.129:0/3754771521 >> v2:172.21.15.196:3300/0 conn(0x7fe98c1608f0 0x7fe98c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2936794056 0 0) 0x7fe978003530 con 0x7fe98c144fc0 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 >> v2:172.21.15.196:3300/0 conn(0x7fe98c1608f0 msgr2=0x7fe98c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 --2- 172.21.15.129:0/3754771521 >> v2:172.21.15.196:3300/0 conn(0x7fe98c1608f0 0x7fe98c06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 >> v1:172.21.15.196:6789/0 conn(0x7fe98c06f390 legacy=0x7fe98c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1600f0 con 0x7fe98c144fc0 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fe988001680 con 0x7fe98c144fc0 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 >> v1:172.21.15.129:6789/0 conn(0x7fe98c144fc0 legacy=0x7fe98c0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9780039c0 0x7fe978003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe978004810 0x7fe978004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9780052f0 0x7fe97800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe97800c190 con 0x7fe978004810 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe97800c340 con 0x7fe9780039c0 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe97800c540 con 0x7fe9780052f0 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe978004810 0x7fe978004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe9917cb640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9780039c0 0x7fe978003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 -- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9780052f0 msgr2=0x7fe97800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9780052f0 0x7fe97800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 -- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9780039c0 msgr2=0x7fe978003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9780039c0 0x7fe978003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe991fcc640 1 -- 172.21.15.129:0/3754771521 reap_dead start 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe990fca640 1 -- 172.21.15.129:0/3754771521 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe978003530 con 0x7fe978004810 2024-04-25T00:46:17.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.975+0000 7fe9917cb640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9780039c0 0x7fe978003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe978004810 0x7fe978004c90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00ba00 tx=0x7fe97c01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcda9514f1b0bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0b3070 con 0x7fe978004810 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c0ae410 con 0x7fe978004810 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe983fff640 1 -- 172.21.15.129:0/3754771521 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00b600 con 0x7fe978004810 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3754771521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe978004810 msgr2=0x7fe978004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe978004810 0x7fe978004c90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00ba00 tx=0x7fe97c01c680 comp rx=0 tx=0).stop 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3754771521 shutdown_connections 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 --2- 172.21.15.129:0/3754771521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe978004810 0x7fe978004c90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3754771521 >> 172.21.15.129:0/3754771521 conn(0x7fe98c067760 msgr2=0x7fe98c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3754771521 shutdown_connections 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3754771521 wait complete. 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 Processor -- start 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- start start 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978004810 0x7fe98c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe98c13a860 0x7fe98c13b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe98c0b0430 0x7fe98c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c070a20 con 0x7fe98c13a860 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c0708a0 con 0x7fe978004810 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe98c070ba0 con 0x7fe98c0b0430 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe98c13a860 0x7fe98c13b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe98c13a860 0x7fe98c13b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38276/0 (socket says 172.21.15.129:38276) 2024-04-25T00:46:17.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 -- 172.21.15.129:0/3494222191 learned_addr learned my addr 172.21.15.129:0/3494222191 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe991fcc640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe98c0b0430 0x7fe98c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 -- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe98c0b0430 msgr2=0x7fe98c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe98c0b0430 0x7fe98c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe9917cb640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978004810 0x7fe98c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 -- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978004810 msgr2=0x7fe98c13cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe978004810 0x7fe98c13cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 -- 172.21.15.129:0/3494222191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c13e3a0 con 0x7fe98c13a860 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe991fcc640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe98c0b0430 0x7fe98c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe990fca640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe98c13a860 0x7fe98c13b170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe97c01cb60 tx=0x7fe978005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0b3040 con 0x7fe98c13a860 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c0aec30 con 0x7fe98c13a860 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c13d300 con 0x7fe98c13a860 2024-04-25T00:46:17.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0ceb10 con 0x7fe98c13a860 2024-04-25T00:46:17.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.979+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c13d8c0 con 0x7fe98c13a860 2024-04-25T00:46:17.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe98c062e50 con 0x7fe98c13a860 2024-04-25T00:46:17.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe97c0c1020 con 0x7fe98c13a860 2024-04-25T00:46:17.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9700678c0 0x7fe970069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fe97c120460 con 0x7fe98c13a860 2024-04-25T00:46:17.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe9917cb640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9700678c0 0x7fe970069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7fe97006fcc0 0x7fe970072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:17.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 --> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe97c001d40 con 0x7fe97006fcc0 2024-04-25T00:46:17.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe97c1208f0 con 0x7fe98c13a860 2024-04-25T00:46:17.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe991fcc640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7fe97006fcc0 0x7fe970072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:17.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe9917cb640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9700678c0 0x7fe970069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe988000fc0 tx=0x7fe988009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe991fcc640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7fe97006fcc0 0x7fe970072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:17.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.983+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== osd.3 v2:172.21.15.129:6810/1553174573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe97c001d40 con 0x7fe97006fcc0 2024-04-25T00:46:17.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 --> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe98c13e160 con 0x7fe97006fcc0 2024-04-25T00:46:17.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe981ffb640 1 -- 172.21.15.129:0/3494222191 <== osd.3 v2:172.21.15.129:6810/1553174573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe98c13e160 con 0x7fe97006fcc0 2024-04-25T00:46:17.999 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:18.000 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:18.000 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7fe97006fcc0 msgr2=0x7fe970072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6810/1553174573,v1:172.21.15.129:6812/1553174573] conn(0x7fe97006fcc0 0x7fe970072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9700678c0 msgr2=0x7fe970069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fe9700678c0 0x7fe970069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe988000fc0 tx=0x7fe988009040 comp rx=0 tx=0).stop 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe98c13a860 msgr2=0x7fe98c13b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 --2- 172.21.15.129:0/3494222191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe98c13a860 0x7fe98c13b170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe97c01cb60 tx=0x7fe978005840 comp rx=0 tx=0).stop 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe991fcc640 1 -- 172.21.15.129:0/3494222191 reap_dead start 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 shutdown_connections 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 >> 172.21.15.129:0/3494222191 conn(0x7fe98c067760 msgr2=0x7fe98c1410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 shutdown_connections 2024-04-25T00:46:18.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:17.999+0000 7fe993a2f640 1 -- 172.21.15.129:0/3494222191 wait complete. 2024-04-25T00:46:18.014 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T00:46:18.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 Processor -- start 2024-04-25T00:46:18.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 -- start start 2024-04-25T00:46:18.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970813f000 0x7f970806b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f970806b750 0x7f970815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9708160090 0x7f97080ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f970806bbb0 con 0x7f9708160090 2024-04-25T00:46:18.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f970813f3e0 con 0x7f970806b750 2024-04-25T00:46:18.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970ebe7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f970815c560 con 0x7f970813f000 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9708160090 0x7f97080ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9708160090 0x7f97080ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38284/0 (socket says 172.21.15.129:38284) 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 -- 172.21.15.129:0/849585340 learned_addr learned my addr 172.21.15.129:0/849585340 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f9707fff640 1 --2- 172.21.15.129:0/849585340 >> v2:172.21.15.196:3300/0 conn(0x7f970806b750 0x7f970815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970c983640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970813f000 0x7f970806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 -- 172.21.15.129:0/849585340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970813f000 msgr2=0x7f970806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970813f000 0x7f970806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 -- 172.21.15.129:0/849585340 >> v2:172.21.15.196:3300/0 conn(0x7f970806b750 msgr2=0x7f970815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 --2- 172.21.15.129:0/849585340 >> v2:172.21.15.196:3300/0 conn(0x7f970806b750 0x7f970815fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970d184640 1 -- 172.21.15.129:0/849585340 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97081608c0 con 0x7f9708160090 2024-04-25T00:46:18.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.123+0000 7f970c983640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970813f000 0x7f970806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:18.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970d184640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9708160090 0x7f97080ceda0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f970806e950 tx=0x7f970001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5924005b325efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f97077fe640 1 -- 172.21.15.129:0/849585340 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000a6500 con 0x7f9708160090 2024-04-25T00:46:18.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f97077fe640 1 -- 172.21.15.129:0/849585340 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97000a66a0 con 0x7f9708160090 2024-04-25T00:46:18.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f97077fe640 1 -- 172.21.15.129:0/849585340 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000a6a70 con 0x7f9708160090 2024-04-25T00:46:18.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/849585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9708160090 msgr2=0x7f97080ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9708160090 0x7f97080ceda0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f970806e950 tx=0x7f970001c670 comp rx=0 tx=0).stop 2024-04-25T00:46:18.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/849585340 shutdown_connections 2024-04-25T00:46:18.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9708160090 0x7f97080ceda0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- 172.21.15.129:0/849585340 >> v2:172.21.15.196:3300/0 conn(0x7f970806b750 0x7f970815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- 172.21.15.129:0/849585340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970813f000 0x7f970806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/849585340 >> 172.21.15.129:0/849585340 conn(0x7f9708070690 msgr2=0x7f9708070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/849585340 shutdown_connections 2024-04-25T00:46:18.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/849585340 wait complete. 2024-04-25T00:46:18.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 Processor -- start 2024-04-25T00:46:18.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- start start 2024-04-25T00:46:18.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970806b750 0x7f9708154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 0x7f97081547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9708160090 0x7f97080abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9708160640 con 0x7f970813f000 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97081604c0 con 0x7f9708160090 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f970815d300 con 0x7f970806b750 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 0x7f97081547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 0x7f97081547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38296/0 (socket says 172.21.15.129:38296) 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 -- 172.21.15.129:0/2258435982 learned_addr learned my addr 172.21.15.129:0/2258435982 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970c983640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970806b750 0x7f9708154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970d184640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9708160090 0x7f97080abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 -- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970806b750 msgr2=0x7f9708154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.134 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970806b750 0x7f9708154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 -- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9708160090 msgr2=0x7f97080abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9708160090 0x7f97080abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 -- 172.21.15.129:0/2258435982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708155180 con 0x7f970813f000 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970d184640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9708160090 0x7f97080abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f9707fff640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 0x7f97081547c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f96f400d800 tx=0x7f96f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4018030 con 0x7f970813f000 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f40096a0 con 0x7f970813f000 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9708155460 con 0x7f970813f000 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4009870 con 0x7f970813f000 2024-04-25T00:46:18.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.127+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9708155a20 con 0x7f970813f000 2024-04-25T00:46:18.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.131+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9708062e50 con 0x7f970813f000 2024-04-25T00:46:18.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.131+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f96f401b070 con 0x7f970813f000 2024-04-25T00:46:18.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.131+0000 7f97057fa640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96ec0678c0 0x7f96ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.131+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f96f4085fb0 con 0x7f970813f000 2024-04-25T00:46:18.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.131+0000 7f970c983640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96ec0678c0 0x7f96ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.131+0000 7f970c983640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96ec0678c0 0x7f96ec069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006fd0 tx=0x7f96fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.135+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f96f4056a40 con 0x7f970813f000 2024-04-25T00:46:18.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.287+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9708145ab0 con 0x7f96ec0678c0 2024-04-25T00:46:18.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.287+0000 7f97057fa640 1 -- 172.21.15.129:0/2258435982 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9708145ab0 con 0x7f96ec0678c0 2024-04-25T00:46:18.289 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T00:46:18.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96ec0678c0 msgr2=0x7f96ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96ec0678c0 0x7f96ec069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006fd0 tx=0x7f96fc008040 comp rx=0 tx=0).stop 2024-04-25T00:46:18.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 msgr2=0x7f97081547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 0x7f97081547c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f96f400d800 tx=0x7f96f400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 shutdown_connections 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f96ec0678c0 0x7f96ec069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9708160090 0x7f97080abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970813f000 0x7f97081547c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 --2- 172.21.15.129:0/2258435982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f970806b750 0x7f9708154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 >> 172.21.15.129:0/2258435982 conn(0x7f9708070690 msgr2=0x7f970815b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 shutdown_connections 2024-04-25T00:46:18.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.291+0000 7f970ebe7640 1 -- 172.21.15.129:0/2258435982 wait complete. 2024-04-25T00:46:18.308 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T00:46:18.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:18.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 Processor -- start 2024-04-25T00:46:18.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 -- start start 2024-04-25T00:46:18.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f88580ad140 0x7f88580d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88580d2620 0x7f885815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88580d2e10 con 0x7f885815f320 2024-04-25T00:46:18.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f885815f700 con 0x7f88580ad140 2024-04-25T00:46:18.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885fd2f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f88580d30f0 con 0x7f88580d2620 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885d2ca640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f88580ad140 0x7f88580d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885d2ca640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f88580ad140 0x7f88580d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38304/0 (socket says 172.21.15.129:38304) 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885d2ca640 1 -- 172.21.15.129:0/1829263372 learned_addr learned my addr 172.21.15.129:0/1829263372 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.423+0000 7f885e2cc640 1 --2- 172.21.15.129:0/1829263372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88580d2620 0x7f885815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885d2ca640 1 -- 172.21.15.129:0/1829263372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88580d2620 msgr2=0x7f885815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885d2ca640 1 --2- 172.21.15.129:0/1829263372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88580d2620 0x7f885815fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885d2ca640 1 -- 172.21.15.129:0/1829263372 >> v1:172.21.15.129:6789/0 conn(0x7f885815f320 legacy=0x7f88580aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885cac9640 1 -- 172.21.15.129:0/1829263372 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425262314 0 0) 0x7f88580d2e10 con 0x7f885815f320 2024-04-25T00:46:18.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885d2ca640 1 -- 172.21.15.129:0/1829263372 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88580d33d0 con 0x7f88580ad140 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885d2ca640 1 --2- 172.21.15.129:0/1829263372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f88580d20e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8848009870 tx=0x7f884801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17548eaa037294b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885cac9640 1 -- 172.21.15.129:0/1829263372 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480b2070 con 0x7f88580ad140 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885cac9640 1 -- 172.21.15.129:0/1829263372 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88480a9070 con 0x7f88580ad140 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885cac9640 1 -- 172.21.15.129:0/1829263372 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480ad880 con 0x7f88580ad140 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 -- 172.21.15.129:0/1829263372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 msgr2=0x7f88580d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 --2- 172.21.15.129:0/1829263372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f88580d20e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8848009870 tx=0x7f884801c560 comp rx=0 tx=0).stop 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 -- 172.21.15.129:0/1829263372 shutdown_connections 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 --2- 172.21.15.129:0/1829263372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88580d2620 0x7f885815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 --2- 172.21.15.129:0/1829263372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f88580d20e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 -- 172.21.15.129:0/1829263372 >> 172.21.15.129:0/1829263372 conn(0x7f8858067240 msgr2=0x7f88580d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 -- 172.21.15.129:0/1829263372 shutdown_connections 2024-04-25T00:46:18.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.427+0000 7f885fd2f640 1 -- 172.21.15.129:0/1829263372 wait complete. 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 Processor -- start 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- start start 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f885813c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88580d2620 0x7f88581449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f885815f320 0x7f88580ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8858160e50 con 0x7f88580ad140 2024-04-25T00:46:18.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8858160cd0 con 0x7f88580d2620 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8858160fd0 con 0x7f885815f320 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f885813c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f885813c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38312/0 (socket says 172.21.15.129:38312) 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 -- 172.21.15.129:0/1872684886 learned_addr learned my addr 172.21.15.129:0/1872684886 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885e2cc640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f885815f320 0x7f88580ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 -- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f885815f320 msgr2=0x7f88580ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f885815f320 0x7f88580ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885d2ca640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88580d2620 0x7f88581449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 -- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88580d2620 msgr2=0x7f88581449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88580d2620 0x7f88581449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 -- 172.21.15.129:0/1872684886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88580aba90 con 0x7f88580ad140 2024-04-25T00:46:18.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885dacb640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f885813c510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8854001ec0 tx=0x7f885400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854017070 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8854003600 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88580abd70 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540039f0 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885813cdf0 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8858062e50 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f885401a070 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.431+0000 7f884e7fc640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f883c0678c0 0x7f883c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f885d2ca640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f883c0678c0 0x7f883c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f8854085880 con 0x7f88580ad140 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f884e7fc640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7f883c06fcc0 0x7f883c072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 --> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f883c072830 con 0x7f883c06fcc0 2024-04-25T00:46:18.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8854085d10 con 0x7f88580ad140 2024-04-25T00:46:18.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f885d2ca640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f883c0678c0 0x7f883c069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88480af000 tx=0x7f8848008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f885e2cc640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7f883c06fcc0 0x7f883c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f885e2cc640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7f883c06fcc0 0x7f883c072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.435+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== osd.4 v2:172.21.15.196:6800/3201055999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f883c072830 con 0x7f883c06fcc0 2024-04-25T00:46:18.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 --> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8858075320 con 0x7f883c06fcc0 2024-04-25T00:46:18.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f884e7fc640 1 -- 172.21.15.129:0/1872684886 <== osd.4 v2:172.21.15.196:6800/3201055999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8858075320 con 0x7f883c06fcc0 2024-04-25T00:46:18.452 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:18.452 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:18.452 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7f883c06fcc0 msgr2=0x7f883c072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.196:6800/3201055999,v1:172.21.15.196:6801/3201055999] conn(0x7f883c06fcc0 0x7f883c072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f883c0678c0 msgr2=0x7f883c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f883c0678c0 0x7f883c069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88480af000 tx=0x7f8848008040 comp rx=0 tx=0).stop 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 msgr2=0x7f885813c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 --2- 172.21.15.129:0/1872684886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88580ad140 0x7f885813c510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8854001ec0 tx=0x7f885400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885e2cc640 1 -- 172.21.15.129:0/1872684886 reap_dead start 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 shutdown_connections 2024-04-25T00:46:18.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 >> 172.21.15.129:0/1872684886 conn(0x7f8858067240 msgr2=0x7f8858074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 shutdown_connections 2024-04-25T00:46:18.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.451+0000 7f885fd2f640 1 -- 172.21.15.129:0/1872684886 wait complete. 2024-04-25T00:46:18.468 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T00:46:18.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 Processor -- start 2024-04-25T00:46:18.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 -- start start 2024-04-25T00:46:18.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbe9c15bf50 0x7fbe9c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0aa0c0 0x7fbe9c0aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9c06d1b0 con 0x7fbe9c0ccb50 2024-04-25T00:46:18.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c06d490 con 0x7fbe9c15bf50 2024-04-25T00:46:18.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbea15dc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c06d770 con 0x7fbe9c0aa0c0 2024-04-25T00:46:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9a7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0aa0c0 0x7fbe9c0aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9affd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbe9c15bf50 0x7fbe9c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9b7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fbe9c0ccb50 0x7fbe9c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.129:40506/0 (socket says 172.21.15.129:40506) 2024-04-25T00:46:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9a7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0aa0c0 0x7fbe9c0aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:49838/0 (socket says 172.21.15.129:49838) 2024-04-25T00:46:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9affd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbe9c15bf50 0x7fbe9c15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.129:35400/0 (socket says 172.21.15.129:35400) 2024-04-25T00:46:18.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9b7fe640 1 -- 172.21.15.129:0/1522668595 learned_addr learned my addr 172.21.15.129:0/1522668595 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:18.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9affd640 1 -- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0aa0c0 msgr2=0x7fbe9c0aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9affd640 1 --2- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0aa0c0 0x7fbe9c0aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9affd640 1 -- 172.21.15.129:0/1522668595 >> v1:172.21.15.196:6789/0 conn(0x7fbe9c0ccb50 legacy=0x7fbe9c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.579+0000 7fbe9affd640 1 -- 172.21.15.129:0/1522668595 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c06da50 con 0x7fbe9c15bf50 2024-04-25T00:46:18.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbe9affd640 1 --2- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c15c330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0097c0 tx=0x7fbe8c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5af052a544612b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbe99ffb640 1 -- 172.21.15.129:0/1522668595 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0aa5f0 con 0x7fbe9c15bf50 2024-04-25T00:46:18.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbe99ffb640 1 -- 172.21.15.129:0/1522668595 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0af070 con 0x7fbe9c15bf50 2024-04-25T00:46:18.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbe99ffb640 1 -- 172.21.15.129:0/1522668595 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0aa9b0 con 0x7fbe9c15bf50 2024-04-25T00:46:18.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 -- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c15bf50 msgr2=0x7fbe9c15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c15c330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0097c0 tx=0x7fbe8c01c470 comp rx=0 tx=0).stop 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 -- 172.21.15.129:0/1522668595 shutdown_connections 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0aa0c0 0x7fbe9c0aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1522668595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c15c330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 -- 172.21.15.129:0/1522668595 >> 172.21.15.129:0/1522668595 conn(0x7fbe9c0b14a0 msgr2=0x7fbe9c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 -- 172.21.15.129:0/1522668595 shutdown_connections 2024-04-25T00:46:18.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 -- 172.21.15.129:0/1522668595 wait complete. 2024-04-25T00:46:18.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.583+0000 7fbea15dc640 1 Processor -- start 2024-04-25T00:46:18.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- start start 2024-04-25T00:46:18.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c0aa0c0 0x7fbe9c1f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0ccb50 0x7fbe9c070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c06cee0 con 0x7fbe9c15bf50 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c06cd60 con 0x7fbe9c0aa0c0 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c169030 con 0x7fbe9c0ccb50 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38314/0 (socket says 172.21.15.129:38314) 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 -- 172.21.15.129:0/1643212274 learned_addr learned my addr 172.21.15.129:0/1643212274 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9a7fc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0ccb50 0x7fbe9c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9affd640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c0aa0c0 0x7fbe9c1f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 -- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0ccb50 msgr2=0x7fbe9c070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0ccb50 0x7fbe9c070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 -- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c0aa0c0 msgr2=0x7fbe9c1f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c0aa0c0 0x7fbe9c1f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 -- 172.21.15.129:0/1643212274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1fba40 con 0x7fbe9c15bf50 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe9b7fe640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c146410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400caa0 tx=0x7fbe9400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94017070 con 0x7fbe9c15bf50 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe9400ac00 con 0x7fbe9c15bf50 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1faec0 con 0x7fbe9c15bf50 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94012620 con 0x7fbe9c15bf50 2024-04-25T00:46:18.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1fb760 con 0x7fbe9c15bf50 2024-04-25T00:46:18.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c1fb2b0 con 0x7fbe9c15bf50 2024-04-25T00:46:18.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.587+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbe9401a070 con 0x7fbe9c15bf50 2024-04-25T00:46:18.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.591+0000 7fbe837fe640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe880678c0 0x7fbe88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.591+0000 7fbe9affd640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe880678c0 0x7fbe88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.591+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fbe94085ec0 con 0x7fbe9c15bf50 2024-04-25T00:46:18.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.591+0000 7fbe9affd640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe880678c0 0x7fbe88069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0ae000 tx=0x7fbe8c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.595+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbe94021020 con 0x7fbe9c15bf50 2024-04-25T00:46:18.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fbe9c062e50 con 0x7fbe880678c0 2024-04-25T00:46:18.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbe837fe640 1 -- 172.21.15.129:0/1643212274 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbe9c062e50 con 0x7fbe880678c0 2024-04-25T00:46:18.749 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T00:46:18.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe880678c0 msgr2=0x7fbe88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe880678c0 0x7fbe88069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0ae000 tx=0x7fbe8c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 msgr2=0x7fbe9c146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c146410 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400caa0 tx=0x7fbe9400cf60 comp rx=0 tx=0).stop 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 shutdown_connections 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbe880678c0 0x7fbe88069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe9c0ccb50 0x7fbe9c070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe9c15bf50 0x7fbe9c146410 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 --2- 172.21.15.129:0/1643212274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe9c0aa0c0 0x7fbe9c1f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 >> 172.21.15.129:0/1643212274 conn(0x7fbe9c0b14a0 msgr2=0x7fbe9c0a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 shutdown_connections 2024-04-25T00:46:18.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.747+0000 7fbea15dc640 1 -- 172.21.15.129:0/1643212274 wait complete. 2024-04-25T00:46:18.766 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T00:46:18.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f6507c30640 1 Processor -- start 2024-04-25T00:46:18.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f6507c30640 1 -- start start 2024-04-25T00:46:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f6507c30640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 0x7f6500161e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f6507c30640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6500071a50 con 0x7f65001623a0 2024-04-25T00:46:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f6507c30640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6500071d30 con 0x7f6500158810 2024-04-25T00:46:18.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f6507c30640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6500162bd0 con 0x7f65000adfa0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65061cd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f65001623a0 0x7f650006ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42128/0 (socket says 172.21.15.129:42128) 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65061cd640 1 -- 172.21.15.129:0/3914764096 learned_addr learned my addr 172.21.15.129:0/3914764096 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 0x7f6500161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662235825 0 0) 0x7f6500071a50 con 0x7f65001623a0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e8003530 con 0x7f65001623a0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2795614778 0 0) 0x7f64e8003530 con 0x7f65001623a0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 msgr2=0x7f6500161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 0x7f6500161e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 >> v1:172.21.15.196:6789/0 conn(0x7f6500158810 legacy=0x7f65000ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6500072010 con 0x7f65001623a0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f64f0001a00 con 0x7f65001623a0 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 >> v1:172.21.15.129:6789/0 conn(0x7f65001623a0 legacy=0x7f650006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 0x7f64e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64e8003f70 0x7f64e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e8004d50 0x7f64e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64e800ba80 con 0x7f64e8003f70 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64e800bc30 con 0x7f64e8004d50 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64e800be30 con 0x7f65000adfa0 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64e8003f70 0x7f64e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65061cd640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e8004d50 0x7f64e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65059cc640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 0x7f64e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 -- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 msgr2=0x7f64e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65000adfa0 0x7f64e80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 -- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e8004d50 msgr2=0x7f64e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e8004d50 0x7f64e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.879+0000 7f65051cb640 1 -- 172.21.15.129:0/3914764096 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e8003530 con 0x7f64e8003f70 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f65051cb640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64e8003f70 0x7f64e8004810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ae0 tx=0x7f64f401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e066d5b380b618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40b2070 con 0x7f64e8003f70 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f40a5550 con 0x7f64e8003f70 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f65049ca640 1 -- 172.21.15.129:0/3914764096 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40096b0 con 0x7f64e8003f70 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- 172.21.15.129:0/3914764096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64e8003f70 msgr2=0x7f64e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 --2- 172.21.15.129:0/3914764096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64e8003f70 0x7f64e8004810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ae0 tx=0x7f64f401ca70 comp rx=0 tx=0).stop 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f65061cd640 1 -- 172.21.15.129:0/3914764096 reap_dead start 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- 172.21.15.129:0/3914764096 shutdown_connections 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- 172.21.15.129:0/3914764096 >> 172.21.15.129:0/3914764096 conn(0x7f6500067760 msgr2=0x7f65000b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- 172.21.15.129:0/3914764096 shutdown_connections 2024-04-25T00:46:18.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- 172.21.15.129:0/3914764096 wait complete. 2024-04-25T00:46:18.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 Processor -- start 2024-04-25T00:46:18.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- start start 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64e8003f70 0x7f650015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6500158bf0 0x7f6500148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6500147870 0x7f650015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6500161e60 con 0x7f6500158bf0 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6500072380 con 0x7f6500147870 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.883+0000 7f6507c30640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6500161fe0 con 0x7f64e8003f70 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6500158bf0 0x7f6500148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6500158bf0 0x7f6500148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38322/0 (socket says 172.21.15.129:38322) 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 -- 172.21.15.129:0/3739946185 learned_addr learned my addr 172.21.15.129:0/3739946185 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65061cd640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6500147870 0x7f650015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65059cc640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64e8003f70 0x7f650015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 -- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64e8003f70 msgr2=0x7f650015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64e8003f70 0x7f650015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 -- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6500147870 msgr2=0x7f650015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6500147870 0x7f650015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 -- 172.21.15.129:0/3739946185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6500159440 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65061cd640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6500147870 0x7f650015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65051cb640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6500158bf0 0x7f6500148400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f64f401c8a0 tx=0x7f64e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40bc020 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f4004020 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6500159720 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40b2040 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6500159ce0 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f650015a0c0 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f64f4003620 con 0x7f6500158bf0 2024-04-25T00:46:18.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f64ec0678f0 0x7f64ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f64f4120f00 con 0x7f6500158bf0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65059cc640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f64ec0678f0 0x7f64ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f64ec06fcf0 0x7f64ec072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 --> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64f4007e50 con 0x7f64ec06fcf0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f64f4121390 con 0x7f6500158bf0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65059cc640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f64ec0678f0 0x7f64ec069db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f64f8001580 tx=0x7f64f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65061cd640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f64ec06fcf0 0x7f64ec072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.887+0000 7f65061cd640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f64ec06fcf0 0x7f64ec072190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:18.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.891+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== osd.5 v2:172.21.15.196:6824/546704147 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64f4007e50 con 0x7f64ec06fcf0 2024-04-25T00:46:18.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 --> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6500062e50 con 0x7f64ec06fcf0 2024-04-25T00:46:18.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f64fe7fc640 1 -- 172.21.15.129:0/3739946185 <== osd.5 v2:172.21.15.196:6824/546704147 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6500062e50 con 0x7f64ec06fcf0 2024-04-25T00:46:18.906 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:18.906 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:18.906 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f64ec06fcf0 msgr2=0x7f64ec072190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.196:6824/546704147,v1:172.21.15.196:6825/546704147] conn(0x7f64ec06fcf0 0x7f64ec072190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f64ec0678f0 msgr2=0x7f64ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f64ec0678f0 0x7f64ec069db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f64f8001580 tx=0x7f64f8009040 comp rx=0 tx=0).stop 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6500158bf0 msgr2=0x7f6500148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 --2- 172.21.15.129:0/3739946185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6500158bf0 0x7f6500148400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f64f401c8a0 tx=0x7f64e8005840 comp rx=0 tx=0).stop 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f65061cd640 1 -- 172.21.15.129:0/3739946185 reap_dead start 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 shutdown_connections 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 >> 172.21.15.129:0/3739946185 conn(0x7f6500067760 msgr2=0x7f650006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 shutdown_connections 2024-04-25T00:46:18.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:18.903+0000 7f6507c30640 1 -- 172.21.15.129:0/3739946185 wait complete. 2024-04-25T00:46:18.920 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T00:46:19.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 Processor -- start 2024-04-25T00:46:19.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 -- start start 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2b100aa0c0 0x7f2b100ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b100cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1006d1b0 con 0x7f2b1015bf50 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1015ca40 con 0x7f2b100aa0c0 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b17b85640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1006d490 con 0x7f2b100aa550 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2b100aa0c0 0x7f2b100ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2b100aa0c0 0x7f2b100ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38336/0 (socket says 172.21.15.129:38336) 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 -- 172.21.15.129:0/3390513497 learned_addr learned my addr 172.21.15.129:0/3390513497 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b16122640 1 --2- 172.21.15.129:0/3390513497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b100cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 -- 172.21.15.129:0/3390513497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 msgr2=0x7f2b100cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 --2- 172.21.15.129:0/3390513497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b100cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 -- 172.21.15.129:0/3390513497 >> v1:172.21.15.196:6789/0 conn(0x7f2b1015bf50 legacy=0x7f2b1015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.027+0000 7f2b15120640 1 -- 172.21.15.129:0/3390513497 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1006d770 con 0x7f2b100aa0c0 2024-04-25T00:46:19.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b15120640 1 --2- 172.21.15.129:0/3390513497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b100aa0c0 0x7f2b100ccb50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009870 tx=0x7f2b0001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ce09eeb0a53ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b1491f640 1 -- 172.21.15.129:0/3390513497 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b000aa430 con 0x7f2b100aa0c0 2024-04-25T00:46:19.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b1491f640 1 -- 172.21.15.129:0/3390513497 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b000af070 con 0x7f2b100aa0c0 2024-04-25T00:46:19.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b1491f640 1 -- 172.21.15.129:0/3390513497 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b000aa7f0 con 0x7f2b100aa0c0 2024-04-25T00:46:19.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- 172.21.15.129:0/3390513497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b100aa0c0 msgr2=0x7f2b100ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 --2- 172.21.15.129:0/3390513497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b100aa0c0 0x7f2b100ccb50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009870 tx=0x7f2b0001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- 172.21.15.129:0/3390513497 shutdown_connections 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 --2- 172.21.15.129:0/3390513497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b100cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 --2- 172.21.15.129:0/3390513497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b100aa0c0 0x7f2b100ccb50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- 172.21.15.129:0/3390513497 >> 172.21.15.129:0/3390513497 conn(0x7f2b100b14a0 msgr2=0x7f2b100b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- 172.21.15.129:0/3390513497 shutdown_connections 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- 172.21.15.129:0/3390513497 wait complete. 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 Processor -- start 2024-04-25T00:46:19.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- start start 2024-04-25T00:46:19.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b100aa0c0 0x7f2b1013d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b1014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 0x7f2b1013ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1006ebe0 con 0x7f2b1015bf50 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1006ea60 con 0x7f2b100aa0c0 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b17b85640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1006ed60 con 0x7f2b100aa550 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 0x7f2b1013ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 0x7f2b1013ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38352/0 (socket says 172.21.15.129:38352) 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 -- 172.21.15.129:0/38776185 learned_addr learned my addr 172.21.15.129:0/38776185 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b15921640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b100aa0c0 0x7f2b1013d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b15120640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b1014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 -- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 msgr2=0x7f2b1014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b1014a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 -- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b100aa0c0 msgr2=0x7f2b1013d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b100aa0c0 0x7f2b1013d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 -- 172.21.15.129:0/38776185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b10153050 con 0x7f2b1015bf50 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b16122640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 0x7f2b1013ddc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b08007e40 tx=0x7f2b0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.031+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08014620 con 0x7f2b1015bf50 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b08019070 con 0x7f2b1015bf50 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1014af90 con 0x7f2b1015bf50 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08014a20 con 0x7f2b1015bf50 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b10146130 con 0x7f2b1015bf50 2024-04-25T00:46:19.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b10062e50 con 0x7f2b1015bf50 2024-04-25T00:46:19.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b08014bc0 con 0x7f2b1015bf50 2024-04-25T00:46:19.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b067fc640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2af40678c0 0x7f2af4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b15921640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2af40678c0 0x7f2af4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f2b08086660 con 0x7f2b1015bf50 2024-04-25T00:46:19.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.035+0000 7f2b15921640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2af40678c0 0x7f2af4069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b10149730 tx=0x7f2b0c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.039+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2b10153050 con 0x7f2b1015bf50 2024-04-25T00:46:19.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.187+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f2b10075320 con 0x7f2af40678c0 2024-04-25T00:46:19.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.187+0000 7f2b067fc640 1 -- 172.21.15.129:0/38776185 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2b10075320 con 0x7f2af40678c0 2024-04-25T00:46:19.189 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T00:46:19.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2af40678c0 msgr2=0x7f2af4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2af40678c0 0x7f2af4069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b10149730 tx=0x7f2b0c008040 comp rx=0 tx=0).stop 2024-04-25T00:46:19.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 msgr2=0x7f2b1013ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 0x7f2b1013ddc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2b08007e40 tx=0x7f2b0800e4b0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 shutdown_connections 2024-04-25T00:46:19.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f2af40678c0 0x7f2af4069d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b100aa550 0x7f2b1014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b1015bf50 0x7f2b1013ddc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 --2- 172.21.15.129:0/38776185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b100aa0c0 0x7f2b1013d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 >> 172.21.15.129:0/38776185 conn(0x7f2b100b14a0 msgr2=0x7f2b100cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 shutdown_connections 2024-04-25T00:46:19.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.191+0000 7f2b17b85640 1 -- 172.21.15.129:0/38776185 wait complete. 2024-04-25T00:46:19.207 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T00:46:19.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:19.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf32f5c640 1 Processor -- start 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf32f5c640 1 -- start start 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf32f5c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf2c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf32f5c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c0ced50 con 0x7fbf2c144fc0 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf32f5c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c15fe10 con 0x7fbf2c06f390 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf32f5c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c1453a0 con 0x7fbf2c1608f0 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf30cf8640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fbf2c144fc0 0x7fbf2c0ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42138/0 (socket says 172.21.15.129:42138) 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf30cf8640 1 -- 172.21.15.129:0/3122308536 learned_addr learned my addr 172.21.15.129:0/3122308536 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf2c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021978995 0 0) 0x7fbf2c0ced50 con 0x7fbf2c144fc0 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf14003530 con 0x7fbf2c144fc0 2024-04-25T00:46:19.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 40224160 0 0) 0x7fbf14003530 con 0x7fbf2c144fc0 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 msgr2=0x7fbf2c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf2c06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 >> v1:172.21.15.196:6789/0 conn(0x7fbf2c06f390 legacy=0x7fbf2c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c1600f0 con 0x7fbf2c144fc0 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf2c06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fbf28001a00 con 0x7fbf2c144fc0 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 >> v1:172.21.15.129:6789/0 conn(0x7fbf2c144fc0 legacy=0x7fbf2c0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf140039f0 0x7fbf14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf14004810 0x7fbf14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf1400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf140054c0 con 0x7fbf14004810 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14005670 con 0x7fbf140039f0 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1400bf10 con 0x7fbf2c1608f0 2024-04-25T00:46:19.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf14004810 0x7fbf14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf30cf8640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf140039f0 0x7fbf14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf314f9640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf1400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 -- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 msgr2=0x7fbf1400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf2c1608f0 0x7fbf1400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 -- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf140039f0 msgr2=0x7fbf14003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf140039f0 0x7fbf14003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf23fff640 1 -- 172.21.15.129:0/3122308536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf14003530 con 0x7fbf14004810 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.327+0000 7fbf30cf8640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf140039f0 0x7fbf14003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf23fff640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf14004810 0x7fbf14004c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098d0 tx=0x7fbf1c01c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4bab665ca20e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00f040 con 0x7fbf14004810 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0b0070 con 0x7fbf14004810 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf237fe640 1 -- 172.21.15.129:0/3122308536 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0ab800 con 0x7fbf14004810 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 -- 172.21.15.129:0/3122308536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf14004810 msgr2=0x7fbf14004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 --2- 172.21.15.129:0/3122308536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf14004810 0x7fbf14004c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098d0 tx=0x7fbf1c01c5d0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf314f9640 1 -- 172.21.15.129:0/3122308536 reap_dead start 2024-04-25T00:46:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 -- 172.21.15.129:0/3122308536 shutdown_connections 2024-04-25T00:46:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 -- 172.21.15.129:0/3122308536 >> 172.21.15.129:0/3122308536 conn(0x7fbf2c067760 msgr2=0x7fbf2c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 -- 172.21.15.129:0/3122308536 shutdown_connections 2024-04-25T00:46:19.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 -- 172.21.15.129:0/3122308536 wait complete. 2024-04-25T00:46:19.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 Processor -- start 2024-04-25T00:46:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.331+0000 7fbf32f5c640 1 -- start start 2024-04-25T00:46:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf14004810 0x7fbf2c15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c143610 0x7fbf2c143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf2c0acde0 0x7fbf2c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c167f60 con 0x7fbf2c0acde0 2024-04-25T00:46:19.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c15fbc0 con 0x7fbf2c143610 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c1680e0 con 0x7fbf14004810 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf2c0acde0 0x7fbf2c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf2c0acde0 0x7fbf2c0ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38364/0 (socket says 172.21.15.129:38364) 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 -- 172.21.15.129:0/2201007441 learned_addr learned my addr 172.21.15.129:0/2201007441 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf30cf8640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf14004810 0x7fbf2c15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 -- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf14004810 msgr2=0x7fbf2c15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf14004810 0x7fbf2c15b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf23fff640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c143610 0x7fbf2c143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 -- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c143610 msgr2=0x7fbf2c143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c143610 0x7fbf2c143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 -- 172.21.15.129:0/2201007441 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0adb50 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf314f9640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf2c0acde0 0x7fbf2c0ad260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0ce1c0 tx=0x7fbf2400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24018070 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf24003600 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c0ade30 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24003a30 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c0ae3f0 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf2c062e50 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf2401b070 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbf180678c0 0x7fbf18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fbf24085320 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf30cf8640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbf180678c0 0x7fbf18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7fbf1806fcc0 0x7fbf18072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 --> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf18072830 con 0x7fbf1806fcc0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf2408c050 con 0x7fbf2c0acde0 2024-04-25T00:46:19.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf30cf8640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbf180678c0 0x7fbf18069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf28000f90 tx=0x7fbf28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.335+0000 7fbf23fff640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7fbf1806fcc0 0x7fbf18072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.339+0000 7fbf23fff640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7fbf1806fcc0 0x7fbf18072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.339+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== osd.6 v2:172.21.15.196:6807/4280234201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf18072830 con 0x7fbf1806fcc0 2024-04-25T00:46:19.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.351+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 --> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbf2c075320 con 0x7fbf1806fcc0 2024-04-25T00:46:19.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.351+0000 7fbf217fa640 1 -- 172.21.15.129:0/2201007441 <== osd.6 v2:172.21.15.196:6807/4280234201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbf2c075320 con 0x7fbf1806fcc0 2024-04-25T00:46:19.355 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:19.355 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:19.355 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.351+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7fbf1806fcc0 msgr2=0x7fbf18072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.351+0000 7fbf32f5c640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.196:6807/4280234201,v1:172.21.15.196:6810/4280234201] conn(0x7fbf1806fcc0 0x7fbf18072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbf180678c0 msgr2=0x7fbf18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fbf180678c0 0x7fbf18069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf28000f90 tx=0x7fbf28009040 comp rx=0 tx=0).stop 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf2c0acde0 msgr2=0x7fbf2c0ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 --2- 172.21.15.129:0/2201007441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf2c0acde0 0x7fbf2c0ad260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0ce1c0 tx=0x7fbf2400d890 comp rx=0 tx=0).stop 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf314f9640 1 -- 172.21.15.129:0/2201007441 reap_dead start 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 shutdown_connections 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 >> 172.21.15.129:0/2201007441 conn(0x7fbf2c067760 msgr2=0x7fbf2c0ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 shutdown_connections 2024-04-25T00:46:19.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.355+0000 7fbf32f5c640 1 -- 172.21.15.129:0/2201007441 wait complete. 2024-04-25T00:46:19.370 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T00:46:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 Processor -- start 2024-04-25T00:46:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 -- start start 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb6d01441b0 0x7fb6d01627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb6d00b6bf0 0x7fb6d0141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d0162cf0 con 0x7fb6d0144590 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d015f6d0 con 0x7fb6d01441b0 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6d5b32640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d015eac0 con 0x7fb6d00b6bf0 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb6d01441b0 0x7fb6d01627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb6d01441b0 0x7fb6d01627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38374/0 (socket says 172.21.15.129:38374) 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 -- 172.21.15.129:0/3552237743 learned_addr learned my addr 172.21.15.129:0/3552237743 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cffff640 1 --2- 172.21.15.129:0/3552237743 >> v2:172.21.15.196:3300/0 conn(0x7fb6d00b6bf0 0x7fb6d0141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 -- 172.21.15.129:0/3552237743 >> v2:172.21.15.196:3300/0 conn(0x7fb6d00b6bf0 msgr2=0x7fb6d0141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 --2- 172.21.15.129:0/3552237743 >> v2:172.21.15.196:3300/0 conn(0x7fb6d00b6bf0 0x7fb6d0141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 -- 172.21.15.129:0/3552237743 >> v1:172.21.15.196:6789/0 conn(0x7fb6d0144590 legacy=0x7fb6d00b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.475+0000 7fb6cf7fe640 1 -- 172.21.15.129:0/3552237743 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d015eda0 con 0x7fb6d01441b0 2024-04-25T00:46:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cf7fe640 1 --2- 172.21.15.129:0/3552237743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d01441b0 0x7fb6d01627b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00711d0 tx=0x7fb6c001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf98367045f782f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6ce7fc640 1 -- 172.21.15.129:0/3552237743 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00a5880 con 0x7fb6d01441b0 2024-04-25T00:46:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6ce7fc640 1 -- 172.21.15.129:0/3552237743 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c00a5a20 con 0x7fb6d01441b0 2024-04-25T00:46:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6ce7fc640 1 -- 172.21.15.129:0/3552237743 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00a5df0 con 0x7fb6d01441b0 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- 172.21.15.129:0/3552237743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d01441b0 msgr2=0x7fb6d01627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/3552237743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d01441b0 0x7fb6d01627b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00711d0 tx=0x7fb6c001c9c0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- 172.21.15.129:0/3552237743 shutdown_connections 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/3552237743 >> v2:172.21.15.196:3300/0 conn(0x7fb6d00b6bf0 0x7fb6d0141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/3552237743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d01441b0 0x7fb6d01627b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- 172.21.15.129:0/3552237743 >> 172.21.15.129:0/3552237743 conn(0x7fb6d00b52d0 msgr2=0x7fb6d00b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- 172.21.15.129:0/3552237743 shutdown_connections 2024-04-25T00:46:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- 172.21.15.129:0/3552237743 wait complete. 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 Processor -- start 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- start start 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d00b6bf0 0x7fb6d00d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6d01441b0 0x7fb6d0154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 0x7fb6d00b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0143c10 con 0x7fb6d0144590 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d0143a90 con 0x7fb6d00b6bf0 2024-04-25T00:46:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6d5b32640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d0142800 con 0x7fb6d01441b0 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 0x7fb6d00b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 0x7fb6d00b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38390/0 (socket says 172.21.15.129:38390) 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 -- 172.21.15.129:0/2535992550 learned_addr learned my addr 172.21.15.129:0/2535992550 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cf7fe640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d00b6bf0 0x7fb6d00d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6ceffd640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6d01441b0 0x7fb6d0154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 -- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6d01441b0 msgr2=0x7fb6d0154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6d01441b0 0x7fb6d0154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 -- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d00b6bf0 msgr2=0x7fb6d00d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d00b6bf0 0x7fb6d00d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.479+0000 7fb6cffff640 1 -- 172.21.15.129:0/2535992550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d013a140 con 0x7fb6d0144590 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6cffff640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 0x7fb6d00b82f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800d5d0 tx=0x7fb6c800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80125f0 con 0x7fb6d0144590 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c8019070 con 0x7fb6d0144590 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d00d2400 con 0x7fb6d0144590 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80129f0 con 0x7fb6d0144590 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d00d26a0 con 0x7fb6d0144590 2024-04-25T00:46:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0062e50 con 0x7fb6d0144590 2024-04-25T00:46:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6c8012b90 con 0x7fb6d0144590 2024-04-25T00:46:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6c7fff640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6b00678c0 0x7fb6b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb6c80860f0 con 0x7fb6d0144590 2024-04-25T00:46:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6cf7fe640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6b00678c0 0x7fb6b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.483+0000 7fb6cf7fe640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6b00678c0 0x7fb6b0069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00ae000 tx=0x7fb6c00a7070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.487+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb6c8056c10 con 0x7fb6d0144590 2024-04-25T00:46:19.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.647+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb6d0141550 con 0x7fb6b00678c0 2024-04-25T00:46:19.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.647+0000 7fb6c7fff640 1 -- 172.21.15.129:0/2535992550 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb6d0141550 con 0x7fb6b00678c0 2024-04-25T00:46:19.650 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T00:46:19.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6b00678c0 msgr2=0x7fb6b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6b00678c0 0x7fb6b0069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00ae000 tx=0x7fb6c00a7070 comp rx=0 tx=0).stop 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 msgr2=0x7fb6d00b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 0x7fb6d00b82f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800d5d0 tx=0x7fb6c800da90 comp rx=0 tx=0).stop 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 shutdown_connections 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb6b00678c0 0x7fb6b0069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6d01441b0 0x7fb6d0154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d0144590 0x7fb6d00b82f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 --2- 172.21.15.129:0/2535992550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d00b6bf0 0x7fb6d00d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 >> 172.21.15.129:0/2535992550 conn(0x7fb6d00b52d0 msgr2=0x7fb6d0074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 shutdown_connections 2024-04-25T00:46:19.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.651+0000 7fb6d5b32640 1 -- 172.21.15.129:0/2535992550 wait complete. 2024-04-25T00:46:19.669 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T00:46:19.669 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:46:19.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 Processor -- start 2024-04-25T00:46:19.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 -- start start 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd6d81627d0 0x7fd6d80725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd6d80b4660 0x7fd6d80cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d80cf010 con 0x7fd6d80b4660 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d8072b00 con 0x7fd6d81627d0 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6de656640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d80ce400 con 0x7fd6d80cf4a0 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd6d80b4660 0x7fd6d80cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd6d80b4660 0x7fd6d80cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38404/0 (socket says 172.21.15.129:38404) 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/246446909 learned_addr learned my addr 172.21.15.129:0/246446909 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6d77fe640 1 --2- 172.21.15.129:0/246446909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6d7fff640 1 --2- 172.21.15.129:0/246446909 >> v2:172.21.15.196:3300/0 conn(0x7fd6d81627d0 0x7fd6d80725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/246446909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 msgr2=0x7fd6d80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 --2- 172.21.15.129:0/246446909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d80cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/246446909 >> v2:172.21.15.196:3300/0 conn(0x7fd6d81627d0 msgr2=0x7fd6d80725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 --2- 172.21.15.129:0/246446909 >> v2:172.21.15.196:3300/0 conn(0x7fd6d81627d0 0x7fd6d80725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.783+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/246446909 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d80ce6e0 con 0x7fd6d80b4660 2024-04-25T00:46:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 --2- 172.21.15.129:0/246446909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d80b4660 0x7fd6d80cc8c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd6d806cf30 tx=0x7fd6c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85397a5cec3d4a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6d6ffd640 1 -- 172.21.15.129:0/246446909 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80b0070 con 0x7fd6d80b4660 2024-04-25T00:46:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6d6ffd640 1 -- 172.21.15.129:0/246446909 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c80ab600 con 0x7fd6d80b4660 2024-04-25T00:46:19.788 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6d6ffd640 1 -- 172.21.15.129:0/246446909 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80ab9d0 con 0x7fd6d80b4660 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- 172.21.15.129:0/246446909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d80b4660 msgr2=0x7fd6d80cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- 172.21.15.129:0/246446909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d80b4660 0x7fd6d80cc8c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd6d806cf30 tx=0x7fd6c801c4f0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- 172.21.15.129:0/246446909 shutdown_connections 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- 172.21.15.129:0/246446909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- 172.21.15.129:0/246446909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d80b4660 0x7fd6d80cc8c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- 172.21.15.129:0/246446909 >> v2:172.21.15.196:3300/0 conn(0x7fd6d81627d0 0x7fd6d80725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- 172.21.15.129:0/246446909 >> 172.21.15.129:0/246446909 conn(0x7fd6d8070690 msgr2=0x7fd6d8070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- 172.21.15.129:0/246446909 shutdown_connections 2024-04-25T00:46:19.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- 172.21.15.129:0/246446909 wait complete. 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 Processor -- start 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- start start 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6d80b4660 0x7fd6d815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d813caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d81627d0 0x7fd6d815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d806def0 con 0x7fd6d81627d0 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d806dd70 con 0x7fd6d80b4660 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6de656640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d806e070 con 0x7fd6d80cf4a0 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d81627d0 0x7fd6d815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d81627d0 0x7fd6d815bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38410/0 (socket says 172.21.15.129:38410) 2024-04-25T00:46:19.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/3366619555 learned_addr learned my addr 172.21.15.129:0/3366619555 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6d7fff640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6d80b4660 0x7fd6d815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6d77fe640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d813caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 msgr2=0x7fd6d813caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6d80cf4a0 0x7fd6d813caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6d80b4660 msgr2=0x7fd6d815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6d80b4660 0x7fd6d815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.787+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/3366619555 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d815cef0 con 0x7fd6d81627d0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6dcbf3640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d81627d0 0x7fd6d815bfe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80ae040 tx=0x7fd6c8019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80ba070 con 0x7fd6d81627d0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c80b4430 con 0x7fd6d81627d0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d815c930 con 0x7fd6d81627d0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80b0040 con 0x7fd6d81627d0 2024-04-25T00:46:19.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d813c2a0 con 0x7fd6d81627d0 2024-04-25T00:46:19.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6d804f510 con 0x7fd6d81627d0 2024-04-25T00:46:19.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd6c80bd070 con 0x7fd6d81627d0 2024-04-25T00:46:19.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd6b80678c0 0x7fd6b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fd6c811ef40 con 0x7fd6d81627d0 2024-04-25T00:46:19.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d7fff640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd6b80678c0 0x7fd6b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7fd6b806fcc0 0x7fd6b8072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 --> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6c800af30 con 0x7fd6b806fcc0 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd6c811f3d0 con 0x7fd6d81627d0 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d77fe640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7fd6b806fcc0 0x7fd6b8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d7fff640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd6b80678c0 0x7fd6b8069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b9d0 tx=0x7fd6d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.791+0000 7fd6d77fe640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7fd6b806fcc0 0x7fd6b8072160 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.795+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== osd.7 v2:172.21.15.196:6806/221824364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6c800af30 con 0x7fd6b806fcc0 2024-04-25T00:46:19.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 --> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd6d8062e50 con 0x7fd6b806fcc0 2024-04-25T00:46:19.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6d4ff9640 1 -- 172.21.15.129:0/3366619555 <== osd.7 v2:172.21.15.196:6806/221824364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd6d8062e50 con 0x7fd6b806fcc0 2024-04-25T00:46:19.809 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T00:46:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:46:19.809 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T00:46:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7fd6b806fcc0 msgr2=0x7fd6b8072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.196:6806/221824364,v1:172.21.15.196:6809/221824364] conn(0x7fd6b806fcc0 0x7fd6b8072160 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd6b80678c0 msgr2=0x7fd6b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd6b80678c0 0x7fd6b8069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b9d0 tx=0x7fd6d0008040 comp rx=0 tx=0).stop 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d81627d0 msgr2=0x7fd6d815bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 --2- 172.21.15.129:0/3366619555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d81627d0 0x7fd6d815bfe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80ae040 tx=0x7fd6c8019290 comp rx=0 tx=0).stop 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6dcbf3640 1 -- 172.21.15.129:0/3366619555 reap_dead start 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 shutdown_connections 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 >> 172.21.15.129:0/3366619555 conn(0x7fd6d8070690 msgr2=0x7fd6d815ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 shutdown_connections 2024-04-25T00:46:19.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.807+0000 7fd6de656640 1 -- 172.21.15.129:0/3366619555 wait complete. 2024-04-25T00:46:19.823 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T00:46:19.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 Processor -- start 2024-04-25T00:46:19.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 -- start start 2024-04-25T00:46:19.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a41441b0 0x7fd9a41627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9a4144590 0x7fd9a40b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd9a40b6a20 0x7fd9a4071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a40b6e80 con 0x7fd9a4144590 2024-04-25T00:46:19.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a415f6d0 con 0x7fd9a40b6a20 2024-04-25T00:46:19.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a99ff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a4162cf0 con 0x7fd9a41441b0 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9a4144590 0x7fd9a40b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9a4144590 0x7fd9a40b64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38426/0 (socket says 172.21.15.129:38426) 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 -- 172.21.15.129:0/3835568219 learned_addr learned my addr 172.21.15.129:0/3835568219 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a2ffd640 1 --2- 172.21.15.129:0/3835568219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a41441b0 0x7fd9a41627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a37fe640 1 --2- 172.21.15.129:0/3835568219 >> v2:172.21.15.196:3300/0 conn(0x7fd9a40b6a20 0x7fd9a4071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 -- 172.21.15.129:0/3835568219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a41441b0 msgr2=0x7fd9a41627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 --2- 172.21.15.129:0/3835568219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a41441b0 0x7fd9a41627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 -- 172.21.15.129:0/3835568219 >> v2:172.21.15.196:3300/0 conn(0x7fd9a40b6a20 msgr2=0x7fd9a4071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 --2- 172.21.15.129:0/3835568219 >> v2:172.21.15.196:3300/0 conn(0x7fd9a40b6a20 0x7fd9a4071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd99bfff640 1 -- 172.21.15.129:0/3835568219 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415eac0 con 0x7fd9a4144590 2024-04-25T00:46:19.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.931+0000 7fd9a37fe640 1 --2- 172.21.15.129:0/3835568219 >> v2:172.21.15.196:3300/0 conn(0x7fd9a40b6a20 0x7fd9a4071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:19.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 --2- 172.21.15.129:0/3835568219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a4144590 0x7fd9a40b64e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4162ea0 tx=0x7fd98c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcd51dae23e753b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a27fc640 1 -- 172.21.15.129:0/3835568219 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0b0070 con 0x7fd9a4144590 2024-04-25T00:46:19.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a27fc640 1 -- 172.21.15.129:0/3835568219 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c0ab600 con 0x7fd9a4144590 2024-04-25T00:46:19.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a27fc640 1 -- 172.21.15.129:0/3835568219 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0ab9d0 con 0x7fd9a4144590 2024-04-25T00:46:19.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- 172.21.15.129:0/3835568219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a4144590 msgr2=0x7fd9a40b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/3835568219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a4144590 0x7fd9a40b64e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4162ea0 tx=0x7fd98c01c230 comp rx=0 tx=0).stop 2024-04-25T00:46:19.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- 172.21.15.129:0/3835568219 shutdown_connections 2024-04-25T00:46:19.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/3835568219 >> v2:172.21.15.196:3300/0 conn(0x7fd9a40b6a20 0x7fd9a4071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/3835568219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a4144590 0x7fd9a40b64e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/3835568219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a41441b0 0x7fd9a41627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- 172.21.15.129:0/3835568219 >> 172.21.15.129:0/3835568219 conn(0x7fd9a40b52d0 msgr2=0x7fd9a40b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:19.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- 172.21.15.129:0/3835568219 shutdown_connections 2024-04-25T00:46:19.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- 172.21.15.129:0/3835568219 wait complete. 2024-04-25T00:46:19.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 Processor -- start 2024-04-25T00:46:19.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- start start 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a40b6a20 0x7fd9a41521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 0x7fd9a413c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a4144590 0x7fd9a413c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4143a90 con 0x7fd9a41441b0 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a415e760 con 0x7fd9a40b6a20 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a99ff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a4143c10 con 0x7fd9a4144590 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 0x7fd9a413c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 0x7fd9a413c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38440/0 (socket says 172.21.15.129:38440) 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 -- 172.21.15.129:0/2042333210 learned_addr learned my addr 172.21.15.129:0/2042333210 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a2ffd640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a40b6a20 0x7fd9a41521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a37fe640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a4144590 0x7fd9a413c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 -- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a4144590 msgr2=0x7fd9a413c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a4144590 0x7fd9a413c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 -- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a40b6a20 msgr2=0x7fd9a41521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a40b6a20 0x7fd9a41521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 -- 172.21.15.129:0/2042333210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415ddf0 con 0x7fd9a41441b0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd9a2ffd640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a40b6a20 0x7fd9a41521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.935+0000 7fd99bfff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 0x7fd9a413c2a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0ae040 tx=0x7fd98c018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0b0040 con 0x7fd9a41441b0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c0bc070 con 0x7fd9a41441b0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a415e0d0 con 0x7fd9a41441b0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0b6880 con 0x7fd9a41441b0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a413e220 con 0x7fd9a41441b0 2024-04-25T00:46:19.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a413e860 con 0x7fd9a41441b0 2024-04-25T00:46:19.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd98c0b6a20 con 0x7fd9a41441b0 2024-04-25T00:46:19.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a0ff9640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9840678c0 0x7fd984069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:19.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fd98c1200f0 con 0x7fd9a41441b0 2024-04-25T00:46:19.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a2ffd640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9840678c0 0x7fd984069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:19.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.939+0000 7fd9a2ffd640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9840678c0 0x7fd984069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd9940097c0 tx=0x7fd994008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:19.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:19.943+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd98c016110 con 0x7fd9a41441b0 2024-04-25T00:46:20.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fd9a4062e50 con 0x7fd9840678c0 2024-04-25T00:46:20.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a0ff9640 1 -- 172.21.15.129:0/2042333210 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd9a4062e50 con 0x7fd9840678c0 2024-04-25T00:46:20.100 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T00:46:20.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9840678c0 msgr2=0x7fd984069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9840678c0 0x7fd984069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd9940097c0 tx=0x7fd994008040 comp rx=0 tx=0).stop 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 msgr2=0x7fd9a413c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 0x7fd9a413c2a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0ae040 tx=0x7fd98c018fd0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 shutdown_connections 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fd9840678c0 0x7fd984069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9a4144590 0x7fd9a413c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9a41441b0 0x7fd9a413c2a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 --2- 172.21.15.129:0/2042333210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a40b6a20 0x7fd9a41521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 >> 172.21.15.129:0/2042333210 conn(0x7fd9a40b52d0 msgr2=0x7fd9a41600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 shutdown_connections 2024-04-25T00:46:20.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.099+0000 7fd9a99ff640 1 -- 172.21.15.129:0/2042333210 wait complete. 2024-04-25T00:46:20.118 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:46:20.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 Processor -- start 2024-04-25T00:46:20.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 -- start start 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69b415bf50 0x7f69b415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f69b40aa0c0 0x7f69b40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69b406d1b0 con 0x7f69b40aa550 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f69b406d490 con 0x7f69b40aa0c0 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69bc00c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f69b406d770 con 0x7f69b415bf50 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69ba5a9640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f69b40aa550 0x7f69b406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:42144/0 (socket says 172.21.15.129:42144) 2024-04-25T00:46:20.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69b95a7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f69b40aa0c0 0x7f69b40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.227+0000 7f69ba5a9640 1 -- 172.21.15.129:0/586036453 learned_addr learned my addr 172.21.15.129:0/586036453 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b9da8640 1 --2- 172.21.15.129:0/586036453 >> v2:172.21.15.196:3300/0 conn(0x7f69b415bf50 0x7f69b415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b8da6640 1 -- 172.21.15.129:0/586036453 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114392422 0 0) 0x7f69b406d1b0 con 0x7f69b40aa550 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b95a7640 1 -- 172.21.15.129:0/586036453 >> v2:172.21.15.196:3300/0 conn(0x7f69b415bf50 msgr2=0x7f69b415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b95a7640 1 --2- 172.21.15.129:0/586036453 >> v2:172.21.15.196:3300/0 conn(0x7f69b415bf50 0x7f69b415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b95a7640 1 -- 172.21.15.129:0/586036453 >> v1:172.21.15.129:6789/0 conn(0x7f69b40aa550 legacy=0x7f69b406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b95a7640 1 -- 172.21.15.129:0/586036453 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b406da50 con 0x7f69b40aa0c0 2024-04-25T00:46:20.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b95a7640 1 --2- 172.21.15.129:0/586036453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa0c0 0x7f69b40ccb50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f69b4069260 tx=0x7f69a401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcac3ce92d206b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:20.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b8da6640 1 -- 172.21.15.129:0/586036453 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40a7070 con 0x7f69b40aa0c0 2024-04-25T00:46:20.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b8da6640 1 -- 172.21.15.129:0/586036453 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a40b0070 con 0x7f69b40aa0c0 2024-04-25T00:46:20.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69b8da6640 1 -- 172.21.15.129:0/586036453 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40ab890 con 0x7f69b40aa0c0 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 -- 172.21.15.129:0/586036453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa0c0 msgr2=0x7f69b40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 --2- 172.21.15.129:0/586036453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa0c0 0x7f69b40ccb50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f69b4069260 tx=0x7f69a401c450 comp rx=0 tx=0).stop 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 -- 172.21.15.129:0/586036453 shutdown_connections 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 --2- 172.21.15.129:0/586036453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa0c0 0x7f69b40ccb50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 --2- 172.21.15.129:0/586036453 >> v2:172.21.15.196:3300/0 conn(0x7f69b415bf50 0x7f69b415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 -- 172.21.15.129:0/586036453 >> 172.21.15.129:0/586036453 conn(0x7f69b40b14a0 msgr2=0x7f69b40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 -- 172.21.15.129:0/586036453 shutdown_connections 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.231+0000 7f69bc00c640 1 -- 172.21.15.129:0/586036453 wait complete. 2024-04-25T00:46:20.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 Processor -- start 2024-04-25T00:46:20.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- start start 2024-04-25T00:46:20.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69b40aa0c0 0x7f69b4157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:20.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 0x7f69b4155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69b415bf50 0x7f69b4145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4169030 con 0x7f69b40aa550 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69b40cd4a0 con 0x7f69b40aa0c0 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f69b41691b0 con 0x7f69b415bf50 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 0x7f69b4155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 0x7f69b4155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38470/0 (socket says 172.21.15.129:38470) 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 -- 172.21.15.129:0/759065706 learned_addr learned my addr 172.21.15.129:0/759065706 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b9da8640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69b40aa0c0 0x7f69b4157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:20.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 -- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69b415bf50 msgr2=0x7f69b4145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69b415bf50 0x7f69b4145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 -- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69b40aa0c0 msgr2=0x7f69b4157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69b40aa0c0 0x7f69b4157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 -- 172.21.15.129:0/759065706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b4146030 con 0x7f69b40aa550 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b95a7640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 0x7f69b4155410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f69a4019060 tx=0x7f69a401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40bf610 con 0x7f69b40aa550 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a40bf7b0 con 0x7f69b40aa550 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b4147ab0 con 0x7f69b40aa550 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40bfb80 con 0x7f69b40aa550 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b41517a0 con 0x7f69b40aa550 2024-04-25T00:46:20.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b4062e50 con 0x7f69b40aa550 2024-04-25T00:46:20.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f69a40bd020 con 0x7f69b40aa550 2024-04-25T00:46:20.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69aa7fc640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f69980678c0 0x7f6998069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:20.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f69a4120720 con 0x7f69b40aa550 2024-04-25T00:46:20.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.235+0000 7f69b9da8640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f69980678c0 0x7f6998069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:20.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.239+0000 7f69b9da8640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f69980678c0 0x7f6998069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69b41f8430 tx=0x7f69b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:20.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.239+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f69b4147ab0 con 0x7f69b40aa550 2024-04-25T00:46:20.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.387+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69b4075320 con 0x7f69980678c0 2024-04-25T00:46:20.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.403+0000 7f69aa7fc640 1 -- 172.21.15.129:0/759065706 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252413 (secure 0 0 0) 0x7f69b4075320 con 0x7f69980678c0 2024-04-25T00:46:20.408 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:20.410 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f69980678c0 msgr2=0x7f6998069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f69980678c0 0x7f6998069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69b41f8430 tx=0x7f69b0008040 comp rx=0 tx=0).stop 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 msgr2=0x7f69b4155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 0x7f69b4155410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f69a4019060 tx=0x7f69a401ac90 comp rx=0 tx=0).stop 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 shutdown_connections 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f69980678c0 0x7f6998069d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69b415bf50 0x7f69b4145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69b40aa550 0x7f69b4155410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 --2- 172.21.15.129:0/759065706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69b40aa0c0 0x7f69b4157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:20.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 >> 172.21.15.129:0/759065706 conn(0x7f69b40b14a0 msgr2=0x7f69b4159550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:20.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 shutdown_connections 2024-04-25T00:46:20.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:20.407+0000 7f69bc00c640 1 -- 172.21.15.129:0/759065706 wait complete. 2024-04-25T00:46:20.426 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-25T00:46:18.665511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31557876,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":181,"num_read_kb":491,"num_write":3411,"num_write_kb":33997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7316,"ondisk_log_size":7316,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333704,"kb_used_data":63496,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754641016,"statfs":{"total":773094113280,"available":772752400384,"internally_reserved":0,"allocated":65019904,"data_stored":63782200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003593"},"pg_stats":[{"pgid":"3.3f","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.546368+0000","last_change":"2024-04-25T00:42:58.690681+0000","last_active":"2024-04-25T00:45:51.546368+0000","last_peered":"2024-04-25T00:45:51.546368+0000","last_clean":"2024-04-25T00:45:51.546368+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:45:51.546368+0000","last_undegraded":"2024-04-25T00:45:51.546368+0000","last_fullsized":"2024-04-25T00:45:51.546368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491744+0000","last_change":"2024-04-25T00:42:58.684582+0000","last_active":"2024-04-25T00:45:51.491744+0000","last_peered":"2024-04-25T00:45:51.491744+0000","last_clean":"2024-04-25T00:45:51.491744+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:45:51.491744+0000","last_undegraded":"2024-04-25T00:45:51.491744+0000","last_fullsized":"2024-04-25T00:45:51.491744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515512+0000","last_change":"2024-04-25T00:42:58.659803+0000","last_active":"2024-04-25T00:45:51.515512+0000","last_peered":"2024-04-25T00:45:51.515512+0000","last_clean":"2024-04-25T00:45:51.515512+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:45:51.515512+0000","last_undegraded":"2024-04-25T00:45:51.515512+0000","last_fullsized":"2024-04-25T00:45:51.515512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'40","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527087+0000","last_change":"2024-04-25T00:42:58.697120+0000","last_active":"2024-04-25T00:45:51.527087+0000","last_peered":"2024-04-25T00:45:51.527087+0000","last_clean":"2024-04-25T00:45:51.527087+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:45:51.527087+0000","last_undegraded":"2024-04-25T00:45:51.527087+0000","last_fullsized":"2024-04-25T00:45:51.527087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.539538+0000","last_change":"2024-04-25T00:42:58.691061+0000","last_active":"2024-04-25T00:45:51.539538+0000","last_peered":"2024-04-25T00:45:51.539538+0000","last_clean":"2024-04-25T00:45:51.539538+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:45:51.539538+0000","last_undegraded":"2024-04-25T00:45:51.539538+0000","last_fullsized":"2024-04-25T00:45:51.539538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527080+0000","last_change":"2024-04-25T00:42:58.701726+0000","last_active":"2024-04-25T00:45:51.527080+0000","last_peered":"2024-04-25T00:45:51.527080+0000","last_clean":"2024-04-25T00:45:51.527080+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:45:51.527080+0000","last_undegraded":"2024-04-25T00:45:51.527080+0000","last_fullsized":"2024-04-25T00:45:51.527080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.545654+0000","last_change":"2024-04-25T00:42:58.701727+0000","last_active":"2024-04-25T00:45:51.545654+0000","last_peered":"2024-04-25T00:45:51.545654+0000","last_clean":"2024-04-25T00:45:51.545654+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:45:51.545654+0000","last_undegraded":"2024-04-25T00:45:51.545654+0000","last_fullsized":"2024-04-25T00:45:51.545654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525802+0000","last_change":"2024-04-25T00:42:58.697396+0000","last_active":"2024-04-25T00:45:51.525802+0000","last_peered":"2024-04-25T00:45:51.525802+0000","last_clean":"2024-04-25T00:45:51.525802+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:45:51.525802+0000","last_undegraded":"2024-04-25T00:45:51.525802+0000","last_fullsized":"2024-04-25T00:45:51.525802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'54","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542405+0000","last_change":"2024-04-25T00:43:39.829539+0000","last_active":"2024-04-25T00:45:51.542405+0000","last_peered":"2024-04-25T00:45:51.542405+0000","last_clean":"2024-04-25T00:45:51.542405+0000","last_became_active":"2024-04-25T00:43:39.828586+0000","last_became_peered":"2024-04-25T00:43:39.828586+0000","last_unstale":"2024-04-25T00:45:51.542405+0000","last_undegraded":"2024-04-25T00:45:51.542405+0000","last_fullsized":"2024-04-25T00:45:51.542405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:31.663001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.540595+0000","last_change":"2024-04-25T00:42:58.694172+0000","last_active":"2024-04-25T00:45:51.540595+0000","last_peered":"2024-04-25T00:45:51.540595+0000","last_clean":"2024-04-25T00:45:51.540595+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:45:51.540595+0000","last_undegraded":"2024-04-25T00:45:51.540595+0000","last_fullsized":"2024-04-25T00:45:51.540595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.528273+0000","last_change":"2024-04-25T00:42:58.698003+0000","last_active":"2024-04-25T00:45:51.528273+0000","last_peered":"2024-04-25T00:45:51.528273+0000","last_clean":"2024-04-25T00:45:51.528273+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:45:51.528273+0000","last_undegraded":"2024-04-25T00:45:51.528273+0000","last_fullsized":"2024-04-25T00:45:51.528273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'44","reported_seq":61,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.532319+0000","last_change":"2024-04-25T00:42:58.666237+0000","last_active":"2024-04-25T00:45:51.532319+0000","last_peered":"2024-04-25T00:45:51.532319+0000","last_clean":"2024-04-25T00:45:51.532319+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:45:51.532319+0000","last_undegraded":"2024-04-25T00:45:51.532319+0000","last_fullsized":"2024-04-25T00:45:51.532319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.512541+0000","last_change":"2024-04-25T00:42:58.704365+0000","last_active":"2024-04-25T00:45:51.512541+0000","last_peered":"2024-04-25T00:45:51.512541+0000","last_clean":"2024-04-25T00:45:51.512541+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:45:51.512541+0000","last_undegraded":"2024-04-25T00:45:51.512541+0000","last_fullsized":"2024-04-25T00:45:51.512541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.519079+0000","last_change":"2024-04-25T00:42:58.688431+0000","last_active":"2024-04-25T00:45:51.519079+0000","last_peered":"2024-04-25T00:45:51.519079+0000","last_clean":"2024-04-25T00:45:51.519079+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:45:51.519079+0000","last_undegraded":"2024-04-25T00:45:51.519079+0000","last_fullsized":"2024-04-25T00:45:51.519079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:54:35.786160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'53","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.495151+0000","last_change":"2024-04-25T00:42:58.678656+0000","last_active":"2024-04-25T00:45:51.495151+0000","last_peered":"2024-04-25T00:45:51.495151+0000","last_clean":"2024-04-25T00:45:51.495151+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:45:51.495151+0000","last_undegraded":"2024-04-25T00:45:51.495151+0000","last_fullsized":"2024-04-25T00:45:51.495151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522025+0000","last_change":"2024-04-25T00:42:58.687851+0000","last_active":"2024-04-25T00:45:51.522025+0000","last_peered":"2024-04-25T00:45:51.522025+0000","last_clean":"2024-04-25T00:45:51.522025+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:45:51.522025+0000","last_undegraded":"2024-04-25T00:45:51.522025+0000","last_fullsized":"2024-04-25T00:45:51.522025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549462+0000","last_change":"2024-04-25T00:42:58.693003+0000","last_active":"2024-04-25T00:45:51.549462+0000","last_peered":"2024-04-25T00:45:51.549462+0000","last_clean":"2024-04-25T00:45:51.549462+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:45:51.549462+0000","last_undegraded":"2024-04-25T00:45:51.549462+0000","last_fullsized":"2024-04-25T00:45:51.549462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:31.215643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.531867+0000","last_change":"2024-04-25T00:42:58.665848+0000","last_active":"2024-04-25T00:45:51.531867+0000","last_peered":"2024-04-25T00:45:51.531867+0000","last_clean":"2024-04-25T00:45:51.531867+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:45:51.531867+0000","last_undegraded":"2024-04-25T00:45:51.531867+0000","last_fullsized":"2024-04-25T00:45:51.531867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533095+0000","last_change":"2024-04-25T00:42:58.691817+0000","last_active":"2024-04-25T00:45:51.533095+0000","last_peered":"2024-04-25T00:45:51.533095+0000","last_clean":"2024-04-25T00:45:51.533095+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:45:51.533095+0000","last_undegraded":"2024-04-25T00:45:51.533095+0000","last_fullsized":"2024-04-25T00:45:51.533095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'62","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515861+0000","last_change":"2024-04-25T00:42:58.671759+0000","last_active":"2024-04-25T00:45:51.515861+0000","last_peered":"2024-04-25T00:45:51.515861+0000","last_clean":"2024-04-25T00:45:51.515861+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:45:51.515861+0000","last_undegraded":"2024-04-25T00:45:51.515861+0000","last_fullsized":"2024-04-25T00:45:51.515861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548225+0000","last_change":"2024-04-25T00:42:58.686323+0000","last_active":"2024-04-25T00:45:51.548225+0000","last_peered":"2024-04-25T00:45:51.548225+0000","last_clean":"2024-04-25T00:45:51.548225+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:45:51.548225+0000","last_undegraded":"2024-04-25T00:45:51.548225+0000","last_fullsized":"2024-04-25T00:45:51.548225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549483+0000","last_change":"2024-04-25T00:42:58.693086+0000","last_active":"2024-04-25T00:45:51.549483+0000","last_peered":"2024-04-25T00:45:51.549483+0000","last_clean":"2024-04-25T00:45:51.549483+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:45:51.549483+0000","last_undegraded":"2024-04-25T00:45:51.549483+0000","last_fullsized":"2024-04-25T00:45:51.549483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510069+0000","last_change":"2024-04-25T00:42:58.707548+0000","last_active":"2024-04-25T00:45:51.510069+0000","last_peered":"2024-04-25T00:45:51.510069+0000","last_clean":"2024-04-25T00:45:51.510069+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:45:51.510069+0000","last_undegraded":"2024-04-25T00:45:51.510069+0000","last_fullsized":"2024-04-25T00:45:51.510069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526220+0000","last_change":"2024-04-25T00:42:58.706369+0000","last_active":"2024-04-25T00:45:51.526220+0000","last_peered":"2024-04-25T00:45:51.526220+0000","last_clean":"2024-04-25T00:45:51.526220+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:45:51.526220+0000","last_undegraded":"2024-04-25T00:45:51.526220+0000","last_fullsized":"2024-04-25T00:45:51.526220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.499400+0000","last_change":"2024-04-25T00:42:58.675611+0000","last_active":"2024-04-25T00:45:51.499400+0000","last_peered":"2024-04-25T00:45:51.499400+0000","last_clean":"2024-04-25T00:45:51.499400+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:45:51.499400+0000","last_undegraded":"2024-04-25T00:45:51.499400+0000","last_fullsized":"2024-04-25T00:45:51.499400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'66","reported_seq":83,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.548312+0000","last_change":"2024-04-25T00:42:58.682611+0000","last_active":"2024-04-25T00:45:51.548312+0000","last_peered":"2024-04-25T00:45:51.548312+0000","last_clean":"2024-04-25T00:45:51.548312+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:45:51.548312+0000","last_undegraded":"2024-04-25T00:45:51.548312+0000","last_fullsized":"2024-04-25T00:45:51.548312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'37","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500102+0000","last_change":"2024-04-25T00:42:58.679682+0000","last_active":"2024-04-25T00:45:51.500102+0000","last_peered":"2024-04-25T00:45:51.500102+0000","last_clean":"2024-04-25T00:45:51.500102+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:45:51.500102+0000","last_undegraded":"2024-04-25T00:45:51.500102+0000","last_fullsized":"2024-04-25T00:45:51.500102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509134+0000","last_change":"2024-04-25T00:42:58.672013+0000","last_active":"2024-04-25T00:45:51.509134+0000","last_peered":"2024-04-25T00:45:51.509134+0000","last_clean":"2024-04-25T00:45:51.509134+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:45:51.509134+0000","last_undegraded":"2024-04-25T00:45:51.509134+0000","last_fullsized":"2024-04-25T00:45:51.509134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:16.846868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'52","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.525040+0000","last_change":"2024-04-25T00:42:58.678592+0000","last_active":"2024-04-25T00:45:51.525040+0000","last_peered":"2024-04-25T00:45:51.525040+0000","last_clean":"2024-04-25T00:45:51.525040+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:45:51.525040+0000","last_undegraded":"2024-04-25T00:45:51.525040+0000","last_fullsized":"2024-04-25T00:45:51.525040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'58","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.500399+0000","last_change":"2024-04-25T00:43:39.818085+0000","last_active":"2024-04-25T00:45:51.500399+0000","last_peered":"2024-04-25T00:45:51.500399+0000","last_clean":"2024-04-25T00:45:51.500399+0000","last_became_active":"2024-04-25T00:43:39.817067+0000","last_became_peered":"2024-04-25T00:43:39.817067+0000","last_unstale":"2024-04-25T00:45:51.500399+0000","last_undegraded":"2024-04-25T00:45:51.500399+0000","last_fullsized":"2024-04-25T00:45:51.500399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"22'59","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.520841+0000","last_change":"2024-04-25T00:42:58.691387+0000","last_active":"2024-04-25T00:45:51.520841+0000","last_peered":"2024-04-25T00:45:51.520841+0000","last_clean":"2024-04-25T00:45:51.520841+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:45:51.520841+0000","last_undegraded":"2024-04-25T00:45:51.520841+0000","last_fullsized":"2024-04-25T00:45:51.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:21.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'69","reported_seq":86,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515490+0000","last_change":"2024-04-25T00:42:58.680317+0000","last_active":"2024-04-25T00:45:51.515490+0000","last_peered":"2024-04-25T00:45:51.515490+0000","last_clean":"2024-04-25T00:45:51.515490+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:45:51.515490+0000","last_undegraded":"2024-04-25T00:45:51.515490+0000","last_fullsized":"2024-04-25T00:45:51.515490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'44","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514482+0000","last_change":"2024-04-25T00:43:39.825468+0000","last_active":"2024-04-25T00:45:51.514482+0000","last_peered":"2024-04-25T00:45:51.514482+0000","last_clean":"2024-04-25T00:45:51.514482+0000","last_became_active":"2024-04-25T00:43:39.824614+0000","last_became_peered":"2024-04-25T00:43:39.824614+0000","last_unstale":"2024-04-25T00:45:51.514482+0000","last_undegraded":"2024-04-25T00:45:51.514482+0000","last_fullsized":"2024-04-25T00:45:51.514482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1500","reported_seq":1519,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.578349+0000","last_change":"2024-04-25T00:42:57.678160+0000","last_active":"2024-04-25T00:45:35.578349+0000","last_peered":"2024-04-25T00:45:35.578349+0000","last_clean":"2024-04-25T00:45:35.578349+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:45:35.578349+0000","last_undegraded":"2024-04-25T00:45:35.578349+0000","last_fullsized":"2024-04-25T00:45:35.578349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":1500,"log_dups_size":0,"ondisk_log_size":1500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1474,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.530748+0000","last_change":"2024-04-25T00:42:58.657167+0000","last_active":"2024-04-25T00:45:51.530748+0000","last_peered":"2024-04-25T00:45:51.530748+0000","last_clean":"2024-04-25T00:45:51.530748+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:45:51.530748+0000","last_undegraded":"2024-04-25T00:45:51.530748+0000","last_fullsized":"2024-04-25T00:45:51.530748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'36","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588170+0000","last_change":"2024-04-25T00:42:57.656747+0000","last_active":"2024-04-25T00:45:35.588170+0000","last_peered":"2024-04-25T00:45:35.588170+0000","last_clean":"2024-04-25T00:45:35.588170+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:45:35.588170+0000","last_undegraded":"2024-04-25T00:45:35.588170+0000","last_fullsized":"2024-04-25T00:45:35.588170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'19","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.962699+0000","last_change":"2024-04-25T00:42:57.652752+0000","last_active":"2024-04-25T00:45:32.962699+0000","last_peered":"2024-04-25T00:45:32.962699+0000","last_clean":"2024-04-25T00:45:32.962699+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:45:32.962699+0000","last_undegraded":"2024-04-25T00:45:32.962699+0000","last_fullsized":"2024-04-25T00:45:32.962699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'65","reported_seq":82,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516989+0000","last_change":"2024-04-25T00:42:58.707891+0000","last_active":"2024-04-25T00:45:51.516989+0000","last_peered":"2024-04-25T00:45:51.516989+0000","last_clean":"2024-04-25T00:45:51.516989+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:45:51.516989+0000","last_undegraded":"2024-04-25T00:45:51.516989+0000","last_fullsized":"2024-04-25T00:45:51.516989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589858+0000","last_change":"2024-04-25T00:42:57.680950+0000","last_active":"2024-04-25T00:45:35.589858+0000","last_peered":"2024-04-25T00:45:35.589858+0000","last_clean":"2024-04-25T00:45:35.589858+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:45:35.589858+0000","last_undegraded":"2024-04-25T00:45:35.589858+0000","last_fullsized":"2024-04-25T00:45:35.589858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'54","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.504832+0000","last_change":"2024-04-25T00:42:58.684632+0000","last_active":"2024-04-25T00:45:51.504832+0000","last_peered":"2024-04-25T00:45:51.504832+0000","last_clean":"2024-04-25T00:45:51.504832+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:45:51.504832+0000","last_undegraded":"2024-04-25T00:45:51.504832+0000","last_fullsized":"2024-04-25T00:45:51.504832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:54.614922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.595446+0000","last_change":"2024-04-25T00:42:57.681141+0000","last_active":"2024-04-25T00:45:35.595446+0000","last_peered":"2024-04-25T00:45:35.595446+0000","last_clean":"2024-04-25T00:45:35.595446+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:45:35.595446+0000","last_undegraded":"2024-04-25T00:45:35.595446+0000","last_fullsized":"2024-04-25T00:45:35.595446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'73","reported_seq":90,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.549377+0000","last_change":"2024-04-25T00:42:58.693186+0000","last_active":"2024-04-25T00:45:51.549377+0000","last_peered":"2024-04-25T00:45:51.549377+0000","last_clean":"2024-04-25T00:45:51.549377+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:45:51.549377+0000","last_undegraded":"2024-04-25T00:45:51.549377+0000","last_fullsized":"2024-04-25T00:45:51.549377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:52.477223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599552+0000","last_change":"2024-04-25T00:42:57.677681+0000","last_active":"2024-04-25T00:45:35.599552+0000","last_peered":"2024-04-25T00:45:35.599552+0000","last_clean":"2024-04-25T00:45:35.599552+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:45:35.599552+0000","last_undegraded":"2024-04-25T00:45:35.599552+0000","last_fullsized":"2024-04-25T00:45:35.599552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521602+0000","last_change":"2024-04-25T00:42:58.700662+0000","last_active":"2024-04-25T00:45:51.521602+0000","last_peered":"2024-04-25T00:45:51.521602+0000","last_clean":"2024-04-25T00:45:51.521602+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:45:51.521602+0000","last_undegraded":"2024-04-25T00:45:51.521602+0000","last_fullsized":"2024-04-25T00:45:51.521602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:55:57.853717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'55","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.142036+0000","last_change":"2024-04-25T00:42:57.682358+0000","last_active":"2024-04-25T00:45:50.142036+0000","last_peered":"2024-04-25T00:45:50.142036+0000","last_clean":"2024-04-25T00:45:50.142036+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:45:50.142036+0000","last_undegraded":"2024-04-25T00:45:50.142036+0000","last_fullsized":"2024-04-25T00:45:50.142036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526491+0000","last_change":"2024-04-25T00:42:58.676325+0000","last_active":"2024-04-25T00:45:51.526491+0000","last_peered":"2024-04-25T00:45:51.526491+0000","last_clean":"2024-04-25T00:45:51.526491+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:45:51.526491+0000","last_undegraded":"2024-04-25T00:45:51.526491+0000","last_fullsized":"2024-04-25T00:45:51.526491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:47.426553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'927","reported_seq":946,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:47.871752+0000","last_change":"2024-04-25T00:42:57.651125+0000","last_active":"2024-04-25T00:45:47.871752+0000","last_peered":"2024-04-25T00:45:47.871752+0000","last_clean":"2024-04-25T00:45:47.871752+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:45:47.871752+0000","last_undegraded":"2024-04-25T00:45:47.871752+0000","last_fullsized":"2024-04-25T00:45:47.871752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":908,"num_write_kb":4392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538578+0000","last_change":"2024-04-25T00:42:58.707890+0000","last_active":"2024-04-25T00:45:51.538578+0000","last_peered":"2024-04-25T00:45:51.538578+0000","last_clean":"2024-04-25T00:45:51.538578+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:45:51.538578+0000","last_undegraded":"2024-04-25T00:45:51.538578+0000","last_fullsized":"2024-04-25T00:45:51.538578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'17","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598995+0000","last_change":"2024-04-25T00:42:57.658383+0000","last_active":"2024-04-25T00:45:35.598995+0000","last_peered":"2024-04-25T00:45:35.598995+0000","last_clean":"2024-04-25T00:45:35.598995+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:45:35.598995+0000","last_undegraded":"2024-04-25T00:45:35.598995+0000","last_fullsized":"2024-04-25T00:45:35.598995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.547835+0000","last_change":"2024-04-25T00:42:58.688614+0000","last_active":"2024-04-25T00:45:51.547835+0000","last_peered":"2024-04-25T00:45:51.547835+0000","last_clean":"2024-04-25T00:45:51.547835+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:45:51.547835+0000","last_undegraded":"2024-04-25T00:45:51.547835+0000","last_fullsized":"2024-04-25T00:45:51.547835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'35","reported_seq":54,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.021801+0000","last_change":"2024-04-25T00:42:57.683000+0000","last_active":"2024-04-25T00:45:39.021801+0000","last_peered":"2024-04-25T00:45:39.021801+0000","last_clean":"2024-04-25T00:45:39.021801+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:45:39.021801+0000","last_undegraded":"2024-04-25T00:45:39.021801+0000","last_fullsized":"2024-04-25T00:45:39.021801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'63","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.542493+0000","last_change":"2024-04-25T00:42:58.686153+0000","last_active":"2024-04-25T00:45:51.542493+0000","last_peered":"2024-04-25T00:45:51.542493+0000","last_clean":"2024-04-25T00:45:51.542493+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:45:51.542493+0000","last_undegraded":"2024-04-25T00:45:51.542493+0000","last_fullsized":"2024-04-25T00:45:51.542493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'30","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:43.055262+0000","last_change":"2024-04-25T00:42:57.663072+0000","last_active":"2024-04-25T00:45:43.055262+0000","last_peered":"2024-04-25T00:45:43.055262+0000","last_clean":"2024-04-25T00:45:43.055262+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:45:43.055262+0000","last_undegraded":"2024-04-25T00:45:43.055262+0000","last_fullsized":"2024-04-25T00:45:43.055262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'49","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.507789+0000","last_change":"2024-04-25T00:42:58.681043+0000","last_active":"2024-04-25T00:45:51.507789+0000","last_peered":"2024-04-25T00:45:51.507789+0000","last_clean":"2024-04-25T00:45:51.507789+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:45:51.507789+0000","last_undegraded":"2024-04-25T00:45:51.507789+0000","last_fullsized":"2024-04-25T00:45:51.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'33","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:46.654507+0000","last_change":"2024-04-25T00:42:57.654123+0000","last_active":"2024-04-25T00:45:46.654507+0000","last_peered":"2024-04-25T00:45:46.654507+0000","last_clean":"2024-04-25T00:45:46.654507+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:45:46.654507+0000","last_undegraded":"2024-04-25T00:45:46.654507+0000","last_fullsized":"2024-04-25T00:45:46.654507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.489200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538189+0000","last_change":"2024-04-25T00:42:58.685629+0000","last_active":"2024-04-25T00:45:51.538189+0000","last_peered":"2024-04-25T00:45:51.538189+0000","last_clean":"2024-04-25T00:45:51.538189+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:45:51.538189+0000","last_undegraded":"2024-04-25T00:45:51.538189+0000","last_fullsized":"2024-04-25T00:45:51.538189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'23","reported_seq":42,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.597426+0000","last_change":"2024-04-25T00:42:57.657899+0000","last_active":"2024-04-25T00:45:35.597426+0000","last_peered":"2024-04-25T00:45:35.597426+0000","last_clean":"2024-04-25T00:45:35.597426+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:45:35.597426+0000","last_undegraded":"2024-04-25T00:45:35.597426+0000","last_fullsized":"2024-04-25T00:45:35.597426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:03.599151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'39","reported_seq":56,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.538292+0000","last_change":"2024-04-25T00:42:58.692114+0000","last_active":"2024-04-25T00:45:51.538292+0000","last_peered":"2024-04-25T00:45:51.538292+0000","last_clean":"2024-04-25T00:45:51.538292+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:45:51.538292+0000","last_undegraded":"2024-04-25T00:45:51.538292+0000","last_fullsized":"2024-04-25T00:45:51.538292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.602720+0000","last_change":"2024-04-25T00:42:57.658479+0000","last_active":"2024-04-25T00:45:35.602720+0000","last_peered":"2024-04-25T00:45:35.602720+0000","last_clean":"2024-04-25T00:45:35.602720+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:45:35.602720+0000","last_undegraded":"2024-04-25T00:45:35.602720+0000","last_fullsized":"2024-04-25T00:45:35.602720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'47","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.541141+0000","last_change":"2024-04-25T00:42:58.696210+0000","last_active":"2024-04-25T00:45:51.541141+0000","last_peered":"2024-04-25T00:45:51.541141+0000","last_clean":"2024-04-25T00:45:51.541141+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:45:51.541141+0000","last_undegraded":"2024-04-25T00:45:51.541141+0000","last_fullsized":"2024-04-25T00:45:51.541141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:54.516306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.603187+0000","last_change":"2024-04-25T00:42:57.657271+0000","last_active":"2024-04-25T00:45:35.603187+0000","last_peered":"2024-04-25T00:45:35.603187+0000","last_clean":"2024-04-25T00:45:35.603187+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:45:35.603187+0000","last_undegraded":"2024-04-25T00:45:35.603187+0000","last_fullsized":"2024-04-25T00:45:35.603187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:46.419915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'43","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.526806+0000","last_change":"2024-04-25T00:43:39.828979+0000","last_active":"2024-04-25T00:45:51.526806+0000","last_peered":"2024-04-25T00:45:51.526806+0000","last_clean":"2024-04-25T00:45:51.526806+0000","last_became_active":"2024-04-25T00:43:39.827957+0000","last_became_peered":"2024-04-25T00:43:39.827957+0000","last_unstale":"2024-04-25T00:45:51.526806+0000","last_undegraded":"2024-04-25T00:45:51.526806+0000","last_fullsized":"2024-04-25T00:45:51.526806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:08:31.750071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'29","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.598652+0000","last_change":"2024-04-25T00:42:57.674173+0000","last_active":"2024-04-25T00:45:35.598652+0000","last_peered":"2024-04-25T00:45:35.598652+0000","last_clean":"2024-04-25T00:45:35.598652+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:45:35.598652+0000","last_undegraded":"2024-04-25T00:45:35.598652+0000","last_fullsized":"2024-04-25T00:45:35.598652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'46","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521754+0000","last_change":"2024-04-25T00:43:39.822064+0000","last_active":"2024-04-25T00:45:51.521754+0000","last_peered":"2024-04-25T00:45:51.521754+0000","last_clean":"2024-04-25T00:45:51.521754+0000","last_became_active":"2024-04-25T00:43:39.821411+0000","last_became_peered":"2024-04-25T00:43:39.821411+0000","last_unstale":"2024-04-25T00:45:51.521754+0000","last_undegraded":"2024-04-25T00:45:51.521754+0000","last_fullsized":"2024-04-25T00:45:51.521754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:43:39.818302+0000","last_change":"2024-04-25T00:42:48.008444+0000","last_active":"2024-04-25T00:43:39.818302+0000","last_peered":"2024-04-25T00:43:39.818302+0000","last_clean":"2024-04-25T00:43:39.818302+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:43:39.818302+0000","last_undegraded":"2024-04-25T00:43:39.818302+0000","last_fullsized":"2024-04-25T00:43:39.818302+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:46.685116+0000","last_clean_scrub_stamp":"2024-04-25T00:42:46.685116+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:40.488731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'20","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.590322+0000","last_change":"2024-04-25T00:42:57.669382+0000","last_active":"2024-04-25T00:45:35.590322+0000","last_peered":"2024-04-25T00:45:35.590322+0000","last_clean":"2024-04-25T00:45:35.590322+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:45:35.590322+0000","last_undegraded":"2024-04-25T00:45:35.590322+0000","last_fullsized":"2024-04-25T00:45:35.590322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'57","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.527964+0000","last_change":"2024-04-25T00:42:58.704580+0000","last_active":"2024-04-25T00:45:51.527964+0000","last_peered":"2024-04-25T00:45:51.527964+0000","last_clean":"2024-04-25T00:45:51.527964+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:45:51.527964+0000","last_undegraded":"2024-04-25T00:45:51.527964+0000","last_fullsized":"2024-04-25T00:45:51.527964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:16.791238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.604044+0000","last_change":"2024-04-25T00:42:57.681628+0000","last_active":"2024-04-25T00:45:35.604044+0000","last_peered":"2024-04-25T00:45:35.604044+0000","last_clean":"2024-04-25T00:45:35.604044+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:45:35.604044+0000","last_undegraded":"2024-04-25T00:45:35.604044+0000","last_fullsized":"2024-04-25T00:45:35.604044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'43","reported_seq":60,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515969+0000","last_change":"2024-04-25T00:42:58.657114+0000","last_active":"2024-04-25T00:45:51.515969+0000","last_peered":"2024-04-25T00:45:51.515969+0000","last_clean":"2024-04-25T00:45:51.515969+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:45:51.515969+0000","last_undegraded":"2024-04-25T00:45:51.515969+0000","last_fullsized":"2024-04-25T00:45:51.515969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'94","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:52.118906+0000","last_change":"2024-04-25T00:42:57.660184+0000","last_active":"2024-04-25T00:45:52.118906+0000","last_peered":"2024-04-25T00:45:52.118906+0000","last_clean":"2024-04-25T00:45:52.118906+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:45:52.118906+0000","last_undegraded":"2024-04-25T00:45:52.118906+0000","last_fullsized":"2024-04-25T00:45:52.118906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357237,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":65,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.515761+0000","last_change":"2024-04-25T00:42:58.694358+0000","last_active":"2024-04-25T00:45:51.515761+0000","last_peered":"2024-04-25T00:45:51.515761+0000","last_clean":"2024-04-25T00:45:51.515761+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:45:51.515761+0000","last_undegraded":"2024-04-25T00:45:51.515761+0000","last_fullsized":"2024-04-25T00:45:51.515761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:41.563039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'24","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599954+0000","last_change":"2024-04-25T00:42:57.667063+0000","last_active":"2024-04-25T00:45:35.599954+0000","last_peered":"2024-04-25T00:45:35.599954+0000","last_clean":"2024-04-25T00:45:35.599954+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:45:35.599954+0000","last_undegraded":"2024-04-25T00:45:35.599954+0000","last_fullsized":"2024-04-25T00:45:35.599954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.522092+0000","last_change":"2024-04-25T00:42:58.664995+0000","last_active":"2024-04-25T00:45:51.522092+0000","last_peered":"2024-04-25T00:45:51.522092+0000","last_clean":"2024-04-25T00:45:51.522092+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:45:51.522092+0000","last_undegraded":"2024-04-25T00:45:51.522092+0000","last_fullsized":"2024-04-25T00:45:51.522092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.603125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'22","reported_seq":41,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.588232+0000","last_change":"2024-04-25T00:42:57.657167+0000","last_active":"2024-04-25T00:45:35.588232+0000","last_peered":"2024-04-25T00:45:35.588232+0000","last_clean":"2024-04-25T00:45:35.588232+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:45:35.588232+0000","last_undegraded":"2024-04-25T00:45:35.588232+0000","last_fullsized":"2024-04-25T00:45:35.588232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:51:25.911908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533188+0000","last_change":"2024-04-25T00:42:58.692607+0000","last_active":"2024-04-25T00:45:51.533188+0000","last_peered":"2024-04-25T00:45:51.533188+0000","last_clean":"2024-04-25T00:45:51.533188+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:45:51.533188+0000","last_undegraded":"2024-04-25T00:45:51.533188+0000","last_fullsized":"2024-04-25T00:45:51.533188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:39:49.108257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.589938+0000","last_change":"2024-04-25T00:42:57.687539+0000","last_active":"2024-04-25T00:45:35.589938+0000","last_peered":"2024-04-25T00:45:35.589938+0000","last_clean":"2024-04-25T00:45:35.589938+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:45:35.589938+0000","last_undegraded":"2024-04-25T00:45:35.589938+0000","last_fullsized":"2024-04-25T00:45:35.589938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:20.230005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'45","reported_seq":62,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.516160+0000","last_change":"2024-04-25T00:42:58.676247+0000","last_active":"2024-04-25T00:45:51.516160+0000","last_peered":"2024-04-25T00:45:51.516160+0000","last_clean":"2024-04-25T00:45:51.516160+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:45:51.516160+0000","last_undegraded":"2024-04-25T00:45:51.516160+0000","last_fullsized":"2024-04-25T00:45:51.516160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'94","reported_seq":113,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:50.752292+0000","last_change":"2024-04-25T00:42:57.651861+0000","last_active":"2024-04-25T00:45:50.752292+0000","last_peered":"2024-04-25T00:45:50.752292+0000","last_clean":"2024-04-25T00:45:50.752292+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:45:50.752292+0000","last_undegraded":"2024-04-25T00:45:50.752292+0000","last_fullsized":"2024-04-25T00:45:50.752292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:08.476570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'51","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.496801+0000","last_change":"2024-04-25T00:42:58.693023+0000","last_active":"2024-04-25T00:45:51.496801+0000","last_peered":"2024-04-25T00:45:51.496801+0000","last_clean":"2024-04-25T00:45:51.496801+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:45:51.496801+0000","last_undegraded":"2024-04-25T00:45:51.496801+0000","last_fullsized":"2024-04-25T00:45:51.496801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:03.642603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'26","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:49.162485+0000","last_change":"2024-04-25T00:42:57.679961+0000","last_active":"2024-04-25T00:45:49.162485+0000","last_peered":"2024-04-25T00:45:49.162485+0000","last_clean":"2024-04-25T00:45:49.162485+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:45:49.162485+0000","last_undegraded":"2024-04-25T00:45:49.162485+0000","last_fullsized":"2024-04-25T00:45:49.162485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.514084+0000","last_change":"2024-04-25T00:42:58.699920+0000","last_active":"2024-04-25T00:45:51.514084+0000","last_peered":"2024-04-25T00:45:51.514084+0000","last_clean":"2024-04-25T00:45:51.514084+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:45:51.514084+0000","last_undegraded":"2024-04-25T00:45:51.514084+0000","last_fullsized":"2024-04-25T00:45:51.514084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:31.260579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'21","reported_seq":40,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.599183+0000","last_change":"2024-04-25T00:42:57.683978+0000","last_active":"2024-04-25T00:45:35.599183+0000","last_peered":"2024-04-25T00:45:35.599183+0000","last_clean":"2024-04-25T00:45:35.599183+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:45:35.599183+0000","last_undegraded":"2024-04-25T00:45:35.599183+0000","last_fullsized":"2024-04-25T00:45:35.599183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:14.723392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.491953+0000","last_change":"2024-04-25T00:42:58.660070+0000","last_active":"2024-04-25T00:45:51.491953+0000","last_peered":"2024-04-25T00:45:51.491953+0000","last_clean":"2024-04-25T00:45:51.491953+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:45:51.491953+0000","last_undegraded":"2024-04-25T00:45:51.491953+0000","last_fullsized":"2024-04-25T00:45:51.491953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:41.496080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'28","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:45.167958+0000","last_change":"2024-04-25T00:42:57.676730+0000","last_active":"2024-04-25T00:45:45.167958+0000","last_peered":"2024-04-25T00:45:45.167958+0000","last_clean":"2024-04-25T00:45:45.167958+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:45:45.167958+0000","last_undegraded":"2024-04-25T00:45:45.167958+0000","last_fullsized":"2024-04-25T00:45:45.167958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:59.990162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'48","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510268+0000","last_change":"2024-04-25T00:42:58.696564+0000","last_active":"2024-04-25T00:45:51.510268+0000","last_peered":"2024-04-25T00:45:51.510268+0000","last_clean":"2024-04-25T00:45:51.510268+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:45:51.510268+0000","last_undegraded":"2024-04-25T00:45:51.510268+0000","last_fullsized":"2024-04-25T00:45:51.510268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:09.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'31","reported_seq":50,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.592850+0000","last_change":"2024-04-25T00:42:57.658457+0000","last_active":"2024-04-25T00:45:35.592850+0000","last_peered":"2024-04-25T00:45:35.592850+0000","last_clean":"2024-04-25T00:45:35.592850+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:45:35.592850+0000","last_undegraded":"2024-04-25T00:45:35.592850+0000","last_fullsized":"2024-04-25T00:45:35.592850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'55","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.521036+0000","last_change":"2024-04-25T00:42:58.696815+0000","last_active":"2024-04-25T00:45:51.521036+0000","last_peered":"2024-04-25T00:45:51.521036+0000","last_clean":"2024-04-25T00:45:51.521036+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:45:51.521036+0000","last_undegraded":"2024-04-25T00:45:51.521036+0000","last_fullsized":"2024-04-25T00:45:51.521036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:53.840447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'508","reported_seq":527,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.577934+0000","last_change":"2024-04-25T00:42:57.681376+0000","last_active":"2024-04-25T00:45:35.577934+0000","last_peered":"2024-04-25T00:45:35.577934+0000","last_clean":"2024-04-25T00:45:35.577934+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:45:35.577934+0000","last_undegraded":"2024-04-25T00:45:35.577934+0000","last_fullsized":"2024-04-25T00:45:35.577934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:05.424535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'42","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.508442+0000","last_change":"2024-04-25T00:42:58.669907+0000","last_active":"2024-04-25T00:45:51.508442+0000","last_peered":"2024-04-25T00:45:51.508442+0000","last_clean":"2024-04-25T00:45:51.508442+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:45:51.508442+0000","last_undegraded":"2024-04-25T00:45:51.508442+0000","last_fullsized":"2024-04-25T00:45:51.508442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:55.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'58","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.535096+0000","last_change":"2024-04-25T00:42:58.696584+0000","last_active":"2024-04-25T00:45:51.535096+0000","last_peered":"2024-04-25T00:45:51.535096+0000","last_clean":"2024-04-25T00:45:51.535096+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:45:51.535096+0000","last_undegraded":"2024-04-25T00:45:51.535096+0000","last_fullsized":"2024-04-25T00:45:51.535096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:47.535104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'16","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:32.973394+0000","last_change":"2024-04-25T00:42:57.688533+0000","last_active":"2024-04-25T00:45:32.973394+0000","last_peered":"2024-04-25T00:45:32.973394+0000","last_clean":"2024-04-25T00:45:32.973394+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:45:32.973394+0000","last_undegraded":"2024-04-25T00:45:32.973394+0000","last_fullsized":"2024-04-25T00:45:32.973394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'50","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.533744+0000","last_change":"2024-04-25T00:42:58.692904+0000","last_active":"2024-04-25T00:45:51.533744+0000","last_peered":"2024-04-25T00:45:51.533744+0000","last_clean":"2024-04-25T00:45:51.533744+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:45:51.533744+0000","last_undegraded":"2024-04-25T00:45:51.533744+0000","last_fullsized":"2024-04-25T00:45:51.533744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:38.705869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'62","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:03.994450+0000","last_change":"2024-04-25T00:42:57.688092+0000","last_active":"2024-04-25T00:46:03.994450+0000","last_peered":"2024-04-25T00:46:03.994450+0000","last_clean":"2024-04-25T00:46:03.994450+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:46:03.994450+0000","last_undegraded":"2024-04-25T00:46:03.994450+0000","last_fullsized":"2024-04-25T00:46:03.994450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:40.489442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5442977,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":5338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'57","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.509543+0000","last_change":"2024-04-25T00:43:39.829056+0000","last_active":"2024-04-25T00:45:51.509543+0000","last_peered":"2024-04-25T00:45:51.509543+0000","last_clean":"2024-04-25T00:45:51.509543+0000","last_became_active":"2024-04-25T00:43:39.828089+0000","last_became_peered":"2024-04-25T00:43:39.828089+0000","last_unstale":"2024-04-25T00:45:51.509543+0000","last_undegraded":"2024-04-25T00:45:51.509543+0000","last_fullsized":"2024-04-25T00:45:51.509543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:23:30.063490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'34","reported_seq":53,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:39.708112+0000","last_change":"2024-04-25T00:42:57.654782+0000","last_active":"2024-04-25T00:45:39.708112+0000","last_peered":"2024-04-25T00:45:39.708112+0000","last_clean":"2024-04-25T00:45:39.708112+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:45:39.708112+0000","last_undegraded":"2024-04-25T00:45:39.708112+0000","last_fullsized":"2024-04-25T00:45:39.708112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:43.596487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'46","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:51.510036+0000","last_change":"2024-04-25T00:42:58.676720+0000","last_active":"2024-04-25T00:45:51.510036+0000","last_peered":"2024-04-25T00:45:51.510036+0000","last_clean":"2024-04-25T00:45:51.510036+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:45:51.510036+0000","last_undegraded":"2024-04-25T00:45:51.510036+0000","last_fullsized":"2024-04-25T00:45:51.510036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:57.624587+0000","last_clean_scrub_stamp":"2024-04-25T00:42:57.624587+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:56.516779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'48","reported_seq":67,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:45:35.581829+0000","last_change":"2024-04-25T00:42:57.679898+0000","last_active":"2024-04-25T00:45:35.581829+0000","last_peered":"2024-04-25T00:45:35.581829+0000","last_clean":"2024-04-25T00:45:35.581829+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:45:35.581829+0000","last_undegraded":"2024-04-25T00:45:35.581829+0000","last_fullsized":"2024-04-25T00:45:35.581829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:42:56.617949+0000","last_clean_scrub_stamp":"2024-04-25T00:42:56.617949+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.666505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":4111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3307,"ondisk_log_size":3307,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":30967508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":3252,"num_write_kb":31581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61997056,"data_stored":61935016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3892,"ondisk_log_size":3892,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57908,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94313932,"statfs":{"total":96636764160,"available":96577466368,"internally_reserved":0,"allocated":18853888,"data_stored":18700145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14069760,"data_stored":13915563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":83396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39604,"kb_used_data":8428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332236,"statfs":{"total":96636764160,"available":96596209664,"internally_reserved":0,"allocated":8630272,"data_stored":8472040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52368,"kb_used_data":12872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319472,"statfs":{"total":96636764160,"available":96583139328,"internally_reserved":0,"allocated":13180928,"data_stored":13024127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":589824,"data_stored":440543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":83466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":9224192,"data_stored":9062920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":357237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12951552,"data_stored":12940821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8388734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13840384,"data_stored":13832257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18030592,"data_stored":18026471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:46:20.438 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.438 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.438 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.439 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.439 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.439 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.439 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.440 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.440 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.440 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.440 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.440 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.441 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.441 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.441 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.441 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.442 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.442 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.442 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.442 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.443 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.443 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.443 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.443 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.443 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.444 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.444 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.444 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.444 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.444 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.444 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.445 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.446 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.446 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.446 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.446 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.446 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.446 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.447 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.447 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.447 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.447 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.447 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.447 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.448 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.449 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.449 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.449 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.449 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.449 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.449 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.450 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.450 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T00:42:46.685116+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=46, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.450 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.450 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.450 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.450 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.451 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.452 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.453 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.453 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.453 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.453 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.453 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.453 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.454 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.454 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.454 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.454 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.454 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.454 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.455 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T00:42:57.624587+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.455 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-25T00:42:56.617949+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=42, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=46, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:46:20.455 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:46:28.301 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:46:28.304 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:46:40.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:46:40.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb25d7f5640 1 Processor -- start 2024-04-25T00:46:40.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb25d7f5640 1 -- start start 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb25d7f5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 0x7fb25806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb25d7f5640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25806d1b0 con 0x7fb25815bf50 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb25d7f5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25806d490 con 0x7fb2580aa0c0 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb25d7f5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb25815ca40 con 0x7fb2580cd260 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb256ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fb25815bf50 0x7fb25815c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55606/0 (socket says 172.21.15.129:55606) 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb256ffd640 1 -- 172.21.15.129:0/1846179007 learned_addr learned my addr 172.21.15.129:0/1846179007 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb2577fe640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 0x7fb25806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085747691 0 0) 0x7fb25806d1b0 con 0x7fb25815bf50 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.587+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb240003530 con 0x7fb25815bf50 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 946426288 0 0) 0x7fb240003530 con 0x7fb25815bf50 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 msgr2=0x7fb25806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:40.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 0x7fb25806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 >> v1:172.21.15.196:6789/0 conn(0x7fb2580aa0c0 legacy=0x7fb2580ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25806d770 con 0x7fb25815bf50 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fb24c001680 con 0x7fb25815bf50 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 >> v1:172.21.15.129:6789/0 conn(0x7fb25815bf50 legacy=0x7fb25815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2400039f0 0x7fb240003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 0x7fb240004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb240004d50 0x7fb2400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb25815ca40 con 0x7fb2400039f0 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb24000bab0 con 0x7fb240004d50 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb24000bcb0 con 0x7fb2580cd260 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2400039f0 0x7fb240003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb2577fe640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb240004d50 0x7fb2400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 -- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 msgr2=0x7fb240004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:40.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2580cd260 0x7fb240004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 -- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb240004d50 msgr2=0x7fb2400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:40.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb240004d50 0x7fb2400051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 -- 172.21.15.129:0/1846179007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb240003530 con 0x7fb2400039f0 2024-04-25T00:46:40.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb256ffd640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2400039f0 0x7fb240003e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb24c001060 tx=0x7fb24c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192b47b8ec36c1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:40.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c007d20 con 0x7fb2400039f0 2024-04-25T00:46:40.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c0a9020 con 0x7fb2400039f0 2024-04-25T00:46:40.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb255ffb640 1 -- 172.21.15.129:0/1846179007 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c005b00 con 0x7fb2400039f0 2024-04-25T00:46:40.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb25d7f5640 1 -- 172.21.15.129:0/1846179007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2400039f0 msgr2=0x7fb240003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/1846179007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2400039f0 0x7fb240003e30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb24c001060 tx=0x7fb24c01cbf0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb2577fe640 1 -- 172.21.15.129:0/1846179007 reap_dead start 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb25d7f5640 1 -- 172.21.15.129:0/1846179007 shutdown_connections 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb25d7f5640 1 -- 172.21.15.129:0/1846179007 >> 172.21.15.129:0/1846179007 conn(0x7fb2580b14a0 msgr2=0x7fb2580b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb25d7f5640 1 -- 172.21.15.129:0/1846179007 shutdown_connections 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.591+0000 7fb25d7f5640 1 -- 172.21.15.129:0/1846179007 wait complete. 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 Processor -- start 2024-04-25T00:46:40.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- start start 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2400039f0 0x7fb2581474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 0x7fb258146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb258156df0 0x7fb258157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb25806ebe0 con 0x7fb2580cf2d0 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb25806ea60 con 0x7fb258156df0 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb25806ed60 con 0x7fb2400039f0 2024-04-25T00:46:40.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 0x7fb258146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 0x7fb258146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55564/0 (socket says 172.21.15.129:55564) 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 -- 172.21.15.129:0/3941635360 learned_addr learned my addr 172.21.15.129:0/3941635360 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb256ffd640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2400039f0 0x7fb2581474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2577fe640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb258156df0 0x7fb258157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 -- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2400039f0 msgr2=0x7fb2581474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2400039f0 0x7fb2581474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 -- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb258156df0 msgr2=0x7fb258157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb258156df0 0x7fb258157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 -- 172.21.15.129:0/3941635360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb258147ba0 con 0x7fb2580cf2d0 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb256ffd640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2400039f0 0x7fb2581474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2577fe640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb258156df0 0x7fb258157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb2567fc640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 0x7fb258146960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c800 tx=0x7fb24800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248003b80 con 0x7fb2580cf2d0 2024-04-25T00:46:40.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248003d40 con 0x7fb2580cf2d0 2024-04-25T00:46:40.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb258147e80 con 0x7fb2580cf2d0 2024-04-25T00:46:40.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24800aa90 con 0x7fb2580cf2d0 2024-04-25T00:46:40.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb258148440 con 0x7fb2580cf2d0 2024-04-25T00:46:40.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.595+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb258062e50 con 0x7fb2580cf2d0 2024-04-25T00:46:40.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.599+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb24800ac30 con 0x7fb2580cf2d0 2024-04-25T00:46:40.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.599+0000 7fb23f7fe640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb2440678f0 0x7fb244069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.599+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb248085d80 con 0x7fb2580cf2d0 2024-04-25T00:46:40.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.599+0000 7fb256ffd640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb2440678f0 0x7fb244069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.599+0000 7fb256ffd640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb2440678f0 0x7fb244069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0aa000 tx=0x7fb24c000ff0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:40.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.603+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb248056810 con 0x7fb2580cf2d0 2024-04-25T00:46:40.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.759+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 --> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb258075320 con 0x7fb2440678f0 2024-04-25T00:46:40.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.775+0000 7fb23f7fe640 1 -- 172.21.15.129:0/3941635360 <== mgr.4108 v2:172.21.15.129:6832/2280039152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252828 (secure 0 0 0) 0x7fb258075320 con 0x7fb2440678f0 2024-04-25T00:46:40.777 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:40.781 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb2440678f0 msgr2=0x7fb244069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb2440678f0 0x7fb244069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0aa000 tx=0x7fb24c000ff0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 msgr2=0x7fb258146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 0x7fb258146960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c800 tx=0x7fb24800ccc0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 shutdown_connections 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb2440678f0 0x7fb244069db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb258156df0 0x7fb258157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2580cf2d0 0x7fb258146960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 --2- 172.21.15.129:0/3941635360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2400039f0 0x7fb2581474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 >> 172.21.15.129:0/3941635360 conn(0x7fb2580b14a0 msgr2=0x7fb2580a9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 shutdown_connections 2024-04-25T00:46:40.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.779+0000 7fb25d7f5640 1 -- 172.21.15.129:0/3941635360 wait complete. 2024-04-25T00:46:40.799 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-04-25T00:46:40.673440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31557876,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":181,"num_read_kb":491,"num_write":3411,"num_write_kb":33997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7344,"ondisk_log_size":7344,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333704,"kb_used_data":63496,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754641016,"statfs":{"total":773094113280,"available":772752400384,"internally_reserved":0,"allocated":65019904,"data_stored":63782200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004315"},"pg_stats":[{"pgid":"3.3f","version":"22'48","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:29.560620+0000","last_change":"2024-04-25T00:46:29.560680+0000","last_active":"2024-04-25T00:46:29.560620+0000","last_peered":"2024-04-25T00:46:29.560620+0000","last_clean":"2024-04-25T00:46:29.560620+0000","last_became_active":"2024-04-25T00:42:58.690064+0000","last_became_peered":"2024-04-25T00:42:58.690064+0000","last_unstale":"2024-04-25T00:46:29.560620+0000","last_undegraded":"2024-04-25T00:46:29.560620+0000","last_fullsized":"2024-04-25T00:46:29.560620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'48","last_scrub_stamp":"2024-04-25T00:46:29.560414+0000","last_deep_scrub":"22'48","last_deep_scrub_stamp":"2024-04-25T00:46:29.560414+0000","last_clean_scrub_stamp":"2024-04-25T00:46:29.560414+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:50.675585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"22'43","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.714406+0000","last_change":"2024-04-25T00:46:27.714548+0000","last_active":"2024-04-25T00:46:27.714406+0000","last_peered":"2024-04-25T00:46:27.714406+0000","last_clean":"2024-04-25T00:46:27.714406+0000","last_became_active":"2024-04-25T00:42:58.683999+0000","last_became_peered":"2024-04-25T00:42:58.683999+0000","last_unstale":"2024-04-25T00:46:27.714406+0000","last_undegraded":"2024-04-25T00:46:27.714406+0000","last_fullsized":"2024-04-25T00:46:27.714406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'43","last_scrub_stamp":"2024-04-25T00:46:27.714157+0000","last_deep_scrub":"22'43","last_deep_scrub_stamp":"2024-04-25T00:46:27.714157+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.714157+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:26.387829+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.320530+0000","last_change":"2024-04-25T00:46:21.320673+0000","last_active":"2024-04-25T00:46:21.320530+0000","last_peered":"2024-04-25T00:46:21.320530+0000","last_clean":"2024-04-25T00:46:21.320530+0000","last_became_active":"2024-04-25T00:42:58.659427+0000","last_became_peered":"2024-04-25T00:42:58.659427+0000","last_unstale":"2024-04-25T00:46:21.320530+0000","last_undegraded":"2024-04-25T00:46:21.320530+0000","last_fullsized":"2024-04-25T00:46:21.320530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:21.320233+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:21.320233+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.320233+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:46.254853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"22'40","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:31.962601+0000","last_change":"2024-04-25T00:46:31.962664+0000","last_active":"2024-04-25T00:46:31.962601+0000","last_peered":"2024-04-25T00:46:31.962601+0000","last_clean":"2024-04-25T00:46:31.962601+0000","last_became_active":"2024-04-25T00:42:58.696572+0000","last_became_peered":"2024-04-25T00:42:58.696572+0000","last_unstale":"2024-04-25T00:46:31.962601+0000","last_undegraded":"2024-04-25T00:46:31.962601+0000","last_fullsized":"2024-04-25T00:46:31.962601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'40","last_scrub_stamp":"2024-04-25T00:46:31.962383+0000","last_deep_scrub":"22'40","last_deep_scrub_stamp":"2024-04-25T00:46:31.962383+0000","last_clean_scrub_stamp":"2024-04-25T00:46:31.962383+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:00:09.481016+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:30.574921+0000","last_change":"2024-04-25T00:46:30.574984+0000","last_active":"2024-04-25T00:46:30.574921+0000","last_peered":"2024-04-25T00:46:30.574921+0000","last_clean":"2024-04-25T00:46:30.574921+0000","last_became_active":"2024-04-25T00:42:58.686273+0000","last_became_peered":"2024-04-25T00:42:58.686273+0000","last_unstale":"2024-04-25T00:46:30.574921+0000","last_undegraded":"2024-04-25T00:46:30.574921+0000","last_fullsized":"2024-04-25T00:46:30.574921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:30.574730+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:30.574730+0000","last_clean_scrub_stamp":"2024-04-25T00:46:30.574730+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:12.074833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"22'51","reported_seq":77,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:32.975389+0000","last_change":"2024-04-25T00:46:32.975454+0000","last_active":"2024-04-25T00:46:32.975389+0000","last_peered":"2024-04-25T00:46:32.975389+0000","last_clean":"2024-04-25T00:46:32.975389+0000","last_became_active":"2024-04-25T00:42:58.701555+0000","last_became_peered":"2024-04-25T00:42:58.701555+0000","last_unstale":"2024-04-25T00:46:32.975389+0000","last_undegraded":"2024-04-25T00:46:32.975389+0000","last_fullsized":"2024-04-25T00:46:32.975389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'51","last_scrub_stamp":"2024-04-25T00:46:32.975182+0000","last_deep_scrub":"22'51","last_deep_scrub_stamp":"2024-04-25T00:46:32.975182+0000","last_clean_scrub_stamp":"2024-04-25T00:46:32.975182+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:14.162413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"22'54","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:34.667314+0000","last_change":"2024-04-25T00:46:34.667380+0000","last_active":"2024-04-25T00:46:34.667314+0000","last_peered":"2024-04-25T00:46:34.667314+0000","last_clean":"2024-04-25T00:46:34.667314+0000","last_became_active":"2024-04-25T00:42:58.701429+0000","last_became_peered":"2024-04-25T00:42:58.701429+0000","last_unstale":"2024-04-25T00:46:34.667314+0000","last_undegraded":"2024-04-25T00:46:34.667314+0000","last_fullsized":"2024-04-25T00:46:34.667314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'54","last_scrub_stamp":"2024-04-25T00:46:34.667119+0000","last_deep_scrub":"22'54","last_deep_scrub_stamp":"2024-04-25T00:46:34.667119+0000","last_clean_scrub_stamp":"2024-04-25T00:46:34.667119+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:15.377601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"22'51","reported_seq":77,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:28.097524+0000","last_change":"2024-04-25T00:46:28.097590+0000","last_active":"2024-04-25T00:46:28.097524+0000","last_peered":"2024-04-25T00:46:28.097524+0000","last_clean":"2024-04-25T00:46:28.097524+0000","last_became_active":"2024-04-25T00:42:58.697065+0000","last_became_peered":"2024-04-25T00:42:58.697065+0000","last_unstale":"2024-04-25T00:46:28.097524+0000","last_undegraded":"2024-04-25T00:46:28.097524+0000","last_fullsized":"2024-04-25T00:46:28.097524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'51","last_scrub_stamp":"2024-04-25T00:46:28.097333+0000","last_deep_scrub":"22'51","last_deep_scrub_stamp":"2024-04-25T00:46:28.097333+0000","last_clean_scrub_stamp":"2024-04-25T00:46:28.097333+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:35.957840+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"22'54","reported_seq":85,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:36.624824+0000","last_change":"2024-04-25T00:46:36.624884+0000","last_active":"2024-04-25T00:46:36.624824+0000","last_peered":"2024-04-25T00:46:36.624824+0000","last_clean":"2024-04-25T00:46:36.624824+0000","last_became_active":"2024-04-25T00:43:39.828586+0000","last_became_peered":"2024-04-25T00:43:39.828586+0000","last_unstale":"2024-04-25T00:46:36.624824+0000","last_undegraded":"2024-04-25T00:46:36.624824+0000","last_fullsized":"2024-04-25T00:46:36.624824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'54","last_scrub_stamp":"2024-04-25T00:46:36.624596+0000","last_deep_scrub":"22'54","last_deep_scrub_stamp":"2024-04-25T00:46:36.624596+0000","last_clean_scrub_stamp":"2024-04-25T00:46:36.624596+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:44.648084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"22'53","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:29.998226+0000","last_change":"2024-04-25T00:46:29.998290+0000","last_active":"2024-04-25T00:46:29.998226+0000","last_peered":"2024-04-25T00:46:29.998226+0000","last_clean":"2024-04-25T00:46:29.998226+0000","last_became_active":"2024-04-25T00:42:58.693903+0000","last_became_peered":"2024-04-25T00:42:58.693903+0000","last_unstale":"2024-04-25T00:46:29.998226+0000","last_undegraded":"2024-04-25T00:46:29.998226+0000","last_fullsized":"2024-04-25T00:46:29.998226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'53","last_scrub_stamp":"2024-04-25T00:46:29.998038+0000","last_deep_scrub":"22'53","last_deep_scrub_stamp":"2024-04-25T00:46:29.998038+0000","last_clean_scrub_stamp":"2024-04-25T00:46:29.998038+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:37.858545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"22'50","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:31.009530+0000","last_change":"2024-04-25T00:46:31.009668+0000","last_active":"2024-04-25T00:46:31.009530+0000","last_peered":"2024-04-25T00:46:31.009530+0000","last_clean":"2024-04-25T00:46:31.009530+0000","last_became_active":"2024-04-25T00:42:58.697694+0000","last_became_peered":"2024-04-25T00:42:58.697694+0000","last_unstale":"2024-04-25T00:46:31.009530+0000","last_undegraded":"2024-04-25T00:46:31.009530+0000","last_fullsized":"2024-04-25T00:46:31.009530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'50","last_scrub_stamp":"2024-04-25T00:46:31.009283+0000","last_deep_scrub":"22'50","last_deep_scrub_stamp":"2024-04-25T00:46:31.009283+0000","last_clean_scrub_stamp":"2024-04-25T00:46:31.009283+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:27.853151+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"22'44","reported_seq":70,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.568721+0000","last_change":"2024-04-25T00:46:22.568884+0000","last_active":"2024-04-25T00:46:22.568721+0000","last_peered":"2024-04-25T00:46:22.568721+0000","last_clean":"2024-04-25T00:46:22.568721+0000","last_became_active":"2024-04-25T00:42:58.665853+0000","last_became_peered":"2024-04-25T00:42:58.665853+0000","last_unstale":"2024-04-25T00:46:22.568721+0000","last_undegraded":"2024-04-25T00:46:22.568721+0000","last_fullsized":"2024-04-25T00:46:22.568721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'44","last_scrub_stamp":"2024-04-25T00:46:22.568472+0000","last_deep_scrub":"22'44","last_deep_scrub_stamp":"2024-04-25T00:46:22.568472+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.568472+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:08.042967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"22'48","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:29.146522+0000","last_change":"2024-04-25T00:46:29.146590+0000","last_active":"2024-04-25T00:46:29.146522+0000","last_peered":"2024-04-25T00:46:29.146522+0000","last_clean":"2024-04-25T00:46:29.146522+0000","last_became_active":"2024-04-25T00:42:58.696581+0000","last_became_peered":"2024-04-25T00:42:58.696581+0000","last_unstale":"2024-04-25T00:46:29.146522+0000","last_undegraded":"2024-04-25T00:46:29.146522+0000","last_fullsized":"2024-04-25T00:46:29.146522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'48","last_scrub_stamp":"2024-04-25T00:46:29.146333+0000","last_deep_scrub":"22'48","last_deep_scrub_stamp":"2024-04-25T00:46:29.146333+0000","last_clean_scrub_stamp":"2024-04-25T00:46:29.146333+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:58.851929+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"22'59","reported_seq":85,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.261412+0000","last_change":"2024-04-25T00:46:24.261480+0000","last_active":"2024-04-25T00:46:24.261412+0000","last_peered":"2024-04-25T00:46:24.261412+0000","last_clean":"2024-04-25T00:46:24.261412+0000","last_became_active":"2024-04-25T00:42:58.688019+0000","last_became_peered":"2024-04-25T00:42:58.688019+0000","last_unstale":"2024-04-25T00:46:24.261412+0000","last_undegraded":"2024-04-25T00:46:24.261412+0000","last_fullsized":"2024-04-25T00:46:24.261412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'59","last_scrub_stamp":"2024-04-25T00:46:24.261220+0000","last_deep_scrub":"22'59","last_deep_scrub_stamp":"2024-04-25T00:46:24.261220+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.261220+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:27.713916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"22'53","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.700897+0000","last_change":"2024-04-25T00:46:24.700959+0000","last_active":"2024-04-25T00:46:24.700897+0000","last_peered":"2024-04-25T00:46:24.700897+0000","last_clean":"2024-04-25T00:46:24.700897+0000","last_became_active":"2024-04-25T00:42:58.678052+0000","last_became_peered":"2024-04-25T00:42:58.678052+0000","last_unstale":"2024-04-25T00:46:24.700897+0000","last_undegraded":"2024-04-25T00:46:24.700897+0000","last_fullsized":"2024-04-25T00:46:24.700897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'53","last_scrub_stamp":"2024-04-25T00:46:24.700705+0000","last_deep_scrub":"22'53","last_deep_scrub_stamp":"2024-04-25T00:46:24.700705+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.700705+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.962798+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.038786+0000","last_change":"2024-04-25T00:46:25.038845+0000","last_active":"2024-04-25T00:46:25.038786+0000","last_peered":"2024-04-25T00:46:25.038786+0000","last_clean":"2024-04-25T00:46:25.038786+0000","last_became_active":"2024-04-25T00:42:58.682961+0000","last_became_peered":"2024-04-25T00:42:58.682961+0000","last_unstale":"2024-04-25T00:46:25.038786+0000","last_undegraded":"2024-04-25T00:46:25.038786+0000","last_fullsized":"2024-04-25T00:46:25.038786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:25.038592+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:25.038592+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.038592+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:58.987278+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:28.025197+0000","last_change":"2024-04-25T00:46:28.025258+0000","last_active":"2024-04-25T00:46:28.025197+0000","last_peered":"2024-04-25T00:46:28.025197+0000","last_clean":"2024-04-25T00:46:28.025197+0000","last_became_active":"2024-04-25T00:42:58.692667+0000","last_became_peered":"2024-04-25T00:42:58.692667+0000","last_unstale":"2024-04-25T00:46:28.025197+0000","last_undegraded":"2024-04-25T00:46:28.025197+0000","last_fullsized":"2024-04-25T00:46:28.025197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:28.025015+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:28.025015+0000","last_clean_scrub_stamp":"2024-04-25T00:46:28.025015+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:24.868883+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"22'58","reported_seq":84,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.288687+0000","last_change":"2024-04-25T00:46:22.288755+0000","last_active":"2024-04-25T00:46:22.288687+0000","last_peered":"2024-04-25T00:46:22.288687+0000","last_clean":"2024-04-25T00:46:22.288687+0000","last_became_active":"2024-04-25T00:42:58.665516+0000","last_became_peered":"2024-04-25T00:42:58.665516+0000","last_unstale":"2024-04-25T00:46:22.288687+0000","last_undegraded":"2024-04-25T00:46:22.288687+0000","last_fullsized":"2024-04-25T00:46:22.288687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'58","last_scrub_stamp":"2024-04-25T00:46:22.288474+0000","last_deep_scrub":"22'58","last_deep_scrub_stamp":"2024-04-25T00:46:22.288474+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.288474+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:24:00.262928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"22'51","reported_seq":77,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:31.623930+0000","last_change":"2024-04-25T00:46:31.623994+0000","last_active":"2024-04-25T00:46:31.623930+0000","last_peered":"2024-04-25T00:46:31.623930+0000","last_clean":"2024-04-25T00:46:31.623930+0000","last_became_active":"2024-04-25T00:42:58.691474+0000","last_became_peered":"2024-04-25T00:42:58.691474+0000","last_unstale":"2024-04-25T00:46:31.623930+0000","last_undegraded":"2024-04-25T00:46:31.623930+0000","last_fullsized":"2024-04-25T00:46:31.623930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'51","last_scrub_stamp":"2024-04-25T00:46:31.623730+0000","last_deep_scrub":"22'51","last_deep_scrub_stamp":"2024-04-25T00:46:31.623730+0000","last_clean_scrub_stamp":"2024-04-25T00:46:31.623730+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.726657+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"22'62","reported_seq":88,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.008035+0000","last_change":"2024-04-25T00:46:26.008100+0000","last_active":"2024-04-25T00:46:26.008035+0000","last_peered":"2024-04-25T00:46:26.008035+0000","last_clean":"2024-04-25T00:46:26.008035+0000","last_became_active":"2024-04-25T00:42:58.671428+0000","last_became_peered":"2024-04-25T00:42:58.671428+0000","last_unstale":"2024-04-25T00:46:26.008035+0000","last_undegraded":"2024-04-25T00:46:26.008035+0000","last_fullsized":"2024-04-25T00:46:26.008035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'62","last_scrub_stamp":"2024-04-25T00:46:26.007834+0000","last_deep_scrub":"22'62","last_deep_scrub_stamp":"2024-04-25T00:46:26.007834+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.007834+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:15.216817+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"22'63","reported_seq":89,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:23.247208+0000","last_change":"2024-04-25T00:46:23.247271+0000","last_active":"2024-04-25T00:46:23.247208+0000","last_peered":"2024-04-25T00:46:23.247208+0000","last_clean":"2024-04-25T00:46:23.247208+0000","last_became_active":"2024-04-25T00:42:58.685932+0000","last_became_peered":"2024-04-25T00:42:58.685932+0000","last_unstale":"2024-04-25T00:46:23.247208+0000","last_undegraded":"2024-04-25T00:46:23.247208+0000","last_fullsized":"2024-04-25T00:46:23.247208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'63","last_scrub_stamp":"2024-04-25T00:46:23.247014+0000","last_deep_scrub":"22'63","last_deep_scrub_stamp":"2024-04-25T00:46:23.247014+0000","last_clean_scrub_stamp":"2024-04-25T00:46:23.247014+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:12.455997+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"22'58","reported_seq":84,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:28.997402+0000","last_change":"2024-04-25T00:46:28.997469+0000","last_active":"2024-04-25T00:46:28.997402+0000","last_peered":"2024-04-25T00:46:28.997402+0000","last_clean":"2024-04-25T00:46:28.997402+0000","last_became_active":"2024-04-25T00:42:58.690368+0000","last_became_peered":"2024-04-25T00:42:58.690368+0000","last_unstale":"2024-04-25T00:46:28.997402+0000","last_undegraded":"2024-04-25T00:46:28.997402+0000","last_fullsized":"2024-04-25T00:46:28.997402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'58","last_scrub_stamp":"2024-04-25T00:46:28.997191+0000","last_deep_scrub":"22'58","last_deep_scrub_stamp":"2024-04-25T00:46:28.997191+0000","last_clean_scrub_stamp":"2024-04-25T00:46:28.997191+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.137928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:31.110196+0000","last_change":"2024-04-25T00:46:31.110264+0000","last_active":"2024-04-25T00:46:31.110196+0000","last_peered":"2024-04-25T00:46:31.110196+0000","last_clean":"2024-04-25T00:46:31.110196+0000","last_became_active":"2024-04-25T00:42:58.707227+0000","last_became_peered":"2024-04-25T00:42:58.707227+0000","last_unstale":"2024-04-25T00:46:31.110196+0000","last_undegraded":"2024-04-25T00:46:31.110196+0000","last_fullsized":"2024-04-25T00:46:31.110196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:31.109967+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:31.109967+0000","last_clean_scrub_stamp":"2024-04-25T00:46:31.109967+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:49.442436+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"22'49","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:33.951122+0000","last_change":"2024-04-25T00:46:33.951247+0000","last_active":"2024-04-25T00:46:33.951122+0000","last_peered":"2024-04-25T00:46:33.951122+0000","last_clean":"2024-04-25T00:46:33.951122+0000","last_became_active":"2024-04-25T00:42:58.706083+0000","last_became_peered":"2024-04-25T00:42:58.706083+0000","last_unstale":"2024-04-25T00:46:33.951122+0000","last_undegraded":"2024-04-25T00:46:33.951122+0000","last_fullsized":"2024-04-25T00:46:33.951122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'49","last_scrub_stamp":"2024-04-25T00:46:33.950843+0000","last_deep_scrub":"22'49","last_deep_scrub_stamp":"2024-04-25T00:46:33.950843+0000","last_clean_scrub_stamp":"2024-04-25T00:46:33.950843+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:53.982534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"22'42","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:23.089800+0000","last_change":"2024-04-25T00:46:23.089861+0000","last_active":"2024-04-25T00:46:23.089800+0000","last_peered":"2024-04-25T00:46:23.089800+0000","last_clean":"2024-04-25T00:46:23.089800+0000","last_became_active":"2024-04-25T00:42:58.675245+0000","last_became_peered":"2024-04-25T00:42:58.675245+0000","last_unstale":"2024-04-25T00:46:23.089800+0000","last_undegraded":"2024-04-25T00:46:23.089800+0000","last_fullsized":"2024-04-25T00:46:23.089800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'42","last_scrub_stamp":"2024-04-25T00:46:23.089599+0000","last_deep_scrub":"22'42","last_deep_scrub_stamp":"2024-04-25T00:46:23.089599+0000","last_clean_scrub_stamp":"2024-04-25T00:46:23.089599+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:50.979441+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'66","reported_seq":92,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:29.005151+0000","last_change":"2024-04-25T00:46:29.005211+0000","last_active":"2024-04-25T00:46:29.005151+0000","last_peered":"2024-04-25T00:46:29.005151+0000","last_clean":"2024-04-25T00:46:29.005151+0000","last_became_active":"2024-04-25T00:42:58.682337+0000","last_became_peered":"2024-04-25T00:42:58.682337+0000","last_unstale":"2024-04-25T00:46:29.005151+0000","last_undegraded":"2024-04-25T00:46:29.005151+0000","last_fullsized":"2024-04-25T00:46:29.005151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'66","last_scrub_stamp":"2024-04-25T00:46:29.004957+0000","last_deep_scrub":"22'66","last_deep_scrub_stamp":"2024-04-25T00:46:29.004957+0000","last_clean_scrub_stamp":"2024-04-25T00:46:29.004957+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:10.632710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"22'37","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.717012+0000","last_change":"2024-04-25T00:46:25.717074+0000","last_active":"2024-04-25T00:46:25.717012+0000","last_peered":"2024-04-25T00:46:25.717012+0000","last_clean":"2024-04-25T00:46:25.717012+0000","last_became_active":"2024-04-25T00:42:58.679000+0000","last_became_peered":"2024-04-25T00:42:58.679000+0000","last_unstale":"2024-04-25T00:46:25.717012+0000","last_undegraded":"2024-04-25T00:46:25.717012+0000","last_fullsized":"2024-04-25T00:46:25.717012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'37","last_scrub_stamp":"2024-04-25T00:46:25.716782+0000","last_deep_scrub":"22'37","last_deep_scrub_stamp":"2024-04-25T00:46:25.716782+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.716782+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:40.508297+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"22'50","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.065415+0000","last_change":"2024-04-25T00:46:22.065479+0000","last_active":"2024-04-25T00:46:22.065415+0000","last_peered":"2024-04-25T00:46:22.065415+0000","last_clean":"2024-04-25T00:46:22.065415+0000","last_became_active":"2024-04-25T00:42:58.671302+0000","last_became_peered":"2024-04-25T00:42:58.671302+0000","last_unstale":"2024-04-25T00:46:22.065415+0000","last_undegraded":"2024-04-25T00:46:22.065415+0000","last_fullsized":"2024-04-25T00:46:22.065415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'50","last_scrub_stamp":"2024-04-25T00:46:22.065205+0000","last_deep_scrub":"22'50","last_deep_scrub_stamp":"2024-04-25T00:46:22.065205+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.065205+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:46.999824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"22'52","reported_seq":78,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.287831+0000","last_change":"2024-04-25T00:46:22.287896+0000","last_active":"2024-04-25T00:46:22.287831+0000","last_peered":"2024-04-25T00:46:22.287831+0000","last_clean":"2024-04-25T00:46:22.287831+0000","last_became_active":"2024-04-25T00:42:58.678194+0000","last_became_peered":"2024-04-25T00:42:58.678194+0000","last_unstale":"2024-04-25T00:46:22.287831+0000","last_undegraded":"2024-04-25T00:46:22.287831+0000","last_fullsized":"2024-04-25T00:46:22.287831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'52","last_scrub_stamp":"2024-04-25T00:46:22.287627+0000","last_deep_scrub":"22'52","last_deep_scrub_stamp":"2024-04-25T00:46:22.287627+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.287627+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:50.177470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"22'58","reported_seq":78,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.279201+0000","last_change":"2024-04-25T00:46:24.279264+0000","last_active":"2024-04-25T00:46:24.279201+0000","last_peered":"2024-04-25T00:46:24.279201+0000","last_clean":"2024-04-25T00:46:24.279201+0000","last_became_active":"2024-04-25T00:43:39.817067+0000","last_became_peered":"2024-04-25T00:43:39.817067+0000","last_unstale":"2024-04-25T00:46:24.279201+0000","last_undegraded":"2024-04-25T00:46:24.279201+0000","last_fullsized":"2024-04-25T00:46:24.279201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'58","last_scrub_stamp":"2024-04-25T00:46:24.279004+0000","last_deep_scrub":"22'58","last_deep_scrub_stamp":"2024-04-25T00:46:24.279004+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.279004+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:25.391293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"22'59","reported_seq":85,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:23.261530+0000","last_change":"2024-04-25T00:46:23.261594+0000","last_active":"2024-04-25T00:46:23.261530+0000","last_peered":"2024-04-25T00:46:23.261530+0000","last_clean":"2024-04-25T00:46:23.261530+0000","last_became_active":"2024-04-25T00:42:58.690796+0000","last_became_peered":"2024-04-25T00:42:58.690796+0000","last_unstale":"2024-04-25T00:46:23.261530+0000","last_undegraded":"2024-04-25T00:46:23.261530+0000","last_fullsized":"2024-04-25T00:46:23.261530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'59","last_scrub_stamp":"2024-04-25T00:46:23.261331+0000","last_deep_scrub":"22'59","last_deep_scrub_stamp":"2024-04-25T00:46:23.261331+0000","last_clean_scrub_stamp":"2024-04-25T00:46:23.261331+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:15.196755+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"22'69","reported_seq":95,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.026080+0000","last_change":"2024-04-25T00:46:27.026138+0000","last_active":"2024-04-25T00:46:27.026080+0000","last_peered":"2024-04-25T00:46:27.026080+0000","last_clean":"2024-04-25T00:46:27.026080+0000","last_became_active":"2024-04-25T00:42:58.679982+0000","last_became_peered":"2024-04-25T00:42:58.679982+0000","last_unstale":"2024-04-25T00:46:27.026080+0000","last_undegraded":"2024-04-25T00:46:27.026080+0000","last_fullsized":"2024-04-25T00:46:27.026080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'69","last_scrub_stamp":"2024-04-25T00:46:27.025889+0000","last_deep_scrub":"22'69","last_deep_scrub_stamp":"2024-04-25T00:46:27.025889+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.025889+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:30.478585+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"22'44","reported_seq":64,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.277847+0000","last_change":"2024-04-25T00:46:26.277907+0000","last_active":"2024-04-25T00:46:26.277847+0000","last_peered":"2024-04-25T00:46:26.277847+0000","last_clean":"2024-04-25T00:46:26.277847+0000","last_became_active":"2024-04-25T00:43:39.824614+0000","last_became_peered":"2024-04-25T00:43:39.824614+0000","last_unstale":"2024-04-25T00:46:26.277847+0000","last_undegraded":"2024-04-25T00:46:26.277847+0000","last_fullsized":"2024-04-25T00:46:26.277847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'44","last_scrub_stamp":"2024-04-25T00:46:26.277655+0000","last_deep_scrub":"22'44","last_deep_scrub_stamp":"2024-04-25T00:46:26.277655+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.277655+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:51.212275+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1501","reported_seq":1529,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.053713+0000","last_change":"2024-04-25T00:46:21.053778+0000","last_active":"2024-04-25T00:46:21.053713+0000","last_peered":"2024-04-25T00:46:21.053713+0000","last_clean":"2024-04-25T00:46:21.053713+0000","last_became_active":"2024-04-25T00:42:57.677773+0000","last_became_peered":"2024-04-25T00:42:57.677773+0000","last_unstale":"2024-04-25T00:46:21.053713+0000","last_undegraded":"2024-04-25T00:46:21.053713+0000","last_fullsized":"2024-04-25T00:46:21.053713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1501","last_scrub_stamp":"2024-04-25T00:46:21.053517+0000","last_deep_scrub":"22'1501","last_deep_scrub_stamp":"2024-04-25T00:46:21.053517+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.053517+0000","objects_scrubbed":1,"log_size":1501,"log_dups_size":0,"ondisk_log_size":1501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:50.383736+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1474,"num_write_kb":4590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"22'51","reported_seq":77,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.330647+0000","last_change":"2024-04-25T00:46:20.330787+0000","last_active":"2024-04-25T00:46:20.330647+0000","last_peered":"2024-04-25T00:46:20.330647+0000","last_clean":"2024-04-25T00:46:20.330647+0000","last_became_active":"2024-04-25T00:42:58.656673+0000","last_became_peered":"2024-04-25T00:42:58.656673+0000","last_unstale":"2024-04-25T00:46:20.330647+0000","last_undegraded":"2024-04-25T00:46:20.330647+0000","last_fullsized":"2024-04-25T00:46:20.330647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'51","last_scrub_stamp":"2024-04-25T00:46:20.330424+0000","last_deep_scrub":"22'51","last_deep_scrub_stamp":"2024-04-25T00:46:20.330424+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.330424+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:01.259189+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'37","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.946194+0000","last_change":"2024-04-25T00:46:20.946260+0000","last_active":"2024-04-25T00:46:20.946194+0000","last_peered":"2024-04-25T00:46:20.946194+0000","last_clean":"2024-04-25T00:46:20.946194+0000","last_became_active":"2024-04-25T00:42:57.656032+0000","last_became_peered":"2024-04-25T00:42:57.656032+0000","last_unstale":"2024-04-25T00:46:20.946194+0000","last_undegraded":"2024-04-25T00:46:20.946194+0000","last_fullsized":"2024-04-25T00:46:20.946194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'37","last_scrub_stamp":"2024-04-25T00:46:20.945978+0000","last_deep_scrub":"22'37","last_deep_scrub_stamp":"2024-04-25T00:46:20.945978+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.945978+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:41.330930+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'19","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:19.025049+0000","last_change":"2024-04-25T00:46:19.025136+0000","last_active":"2024-04-25T00:46:19.025049+0000","last_peered":"2024-04-25T00:46:19.025049+0000","last_clean":"2024-04-25T00:46:19.025049+0000","last_became_active":"2024-04-25T00:42:57.652294+0000","last_became_peered":"2024-04-25T00:42:57.652294+0000","last_unstale":"2024-04-25T00:46:19.025049+0000","last_undegraded":"2024-04-25T00:46:19.025049+0000","last_fullsized":"2024-04-25T00:46:19.025049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'19","last_scrub_stamp":"2024-04-25T00:46:19.024839+0000","last_deep_scrub":"22'19","last_deep_scrub_stamp":"2024-04-25T00:46:19.024839+0000","last_clean_scrub_stamp":"2024-04-25T00:46:19.024839+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:39.409792+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"22'65","reported_seq":91,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:32.076596+0000","last_change":"2024-04-25T00:46:32.076659+0000","last_active":"2024-04-25T00:46:32.076596+0000","last_peered":"2024-04-25T00:46:32.076596+0000","last_clean":"2024-04-25T00:46:32.076596+0000","last_became_active":"2024-04-25T00:42:58.707376+0000","last_became_peered":"2024-04-25T00:42:58.707376+0000","last_unstale":"2024-04-25T00:46:32.076596+0000","last_undegraded":"2024-04-25T00:46:32.076596+0000","last_fullsized":"2024-04-25T00:46:32.076596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'65","last_scrub_stamp":"2024-04-25T00:46:32.076403+0000","last_deep_scrub":"22'65","last_deep_scrub_stamp":"2024-04-25T00:46:32.076403+0000","last_clean_scrub_stamp":"2024-04-25T00:46:32.076403+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:13.576507+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'20","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.991534+0000","last_change":"2024-04-25T00:46:22.991677+0000","last_active":"2024-04-25T00:46:22.991534+0000","last_peered":"2024-04-25T00:46:22.991534+0000","last_clean":"2024-04-25T00:46:22.991534+0000","last_became_active":"2024-04-25T00:42:57.680357+0000","last_became_peered":"2024-04-25T00:42:57.680357+0000","last_unstale":"2024-04-25T00:46:22.991534+0000","last_undegraded":"2024-04-25T00:46:22.991534+0000","last_fullsized":"2024-04-25T00:46:22.991534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'20","last_scrub_stamp":"2024-04-25T00:46:22.991237+0000","last_deep_scrub":"22'20","last_deep_scrub_stamp":"2024-04-25T00:46:22.991237+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.991237+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:12.200220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"22'54","reported_seq":80,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.761388+0000","last_change":"2024-04-25T00:46:26.761455+0000","last_active":"2024-04-25T00:46:26.761388+0000","last_peered":"2024-04-25T00:46:26.761388+0000","last_clean":"2024-04-25T00:46:26.761388+0000","last_became_active":"2024-04-25T00:42:58.678136+0000","last_became_peered":"2024-04-25T00:42:58.678136+0000","last_unstale":"2024-04-25T00:46:26.761388+0000","last_undegraded":"2024-04-25T00:46:26.761388+0000","last_fullsized":"2024-04-25T00:46:26.761388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'54","last_scrub_stamp":"2024-04-25T00:46:26.761180+0000","last_deep_scrub":"22'54","last_deep_scrub_stamp":"2024-04-25T00:46:26.761180+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.761180+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:23.558134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'23","reported_seq":51,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.017227+0000","last_change":"2024-04-25T00:46:24.017291+0000","last_active":"2024-04-25T00:46:24.017227+0000","last_peered":"2024-04-25T00:46:24.017227+0000","last_clean":"2024-04-25T00:46:24.017227+0000","last_became_active":"2024-04-25T00:42:57.680593+0000","last_became_peered":"2024-04-25T00:42:57.680593+0000","last_unstale":"2024-04-25T00:46:24.017227+0000","last_undegraded":"2024-04-25T00:46:24.017227+0000","last_fullsized":"2024-04-25T00:46:24.017227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'23","last_scrub_stamp":"2024-04-25T00:46:24.017032+0000","last_deep_scrub":"22'23","last_deep_scrub_stamp":"2024-04-25T00:46:24.017032+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.017032+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:48.951651+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"22'73","reported_seq":99,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:29.988742+0000","last_change":"2024-04-25T00:46:29.988803+0000","last_active":"2024-04-25T00:46:29.988742+0000","last_peered":"2024-04-25T00:46:29.988742+0000","last_clean":"2024-04-25T00:46:29.988742+0000","last_became_active":"2024-04-25T00:42:58.692699+0000","last_became_peered":"2024-04-25T00:42:58.692699+0000","last_unstale":"2024-04-25T00:46:29.988742+0000","last_undegraded":"2024-04-25T00:46:29.988742+0000","last_fullsized":"2024-04-25T00:46:29.988742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'73","last_scrub_stamp":"2024-04-25T00:46:29.988548+0000","last_deep_scrub":"22'73","last_deep_scrub_stamp":"2024-04-25T00:46:29.988548+0000","last_clean_scrub_stamp":"2024-04-25T00:46:29.988548+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:01:51.103719+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'24","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.966390+0000","last_change":"2024-04-25T00:46:21.966455+0000","last_active":"2024-04-25T00:46:21.966390+0000","last_peered":"2024-04-25T00:46:21.966390+0000","last_clean":"2024-04-25T00:46:21.966390+0000","last_became_active":"2024-04-25T00:42:57.676956+0000","last_became_peered":"2024-04-25T00:42:57.676956+0000","last_unstale":"2024-04-25T00:46:21.966390+0000","last_undegraded":"2024-04-25T00:46:21.966390+0000","last_fullsized":"2024-04-25T00:46:21.966390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'24","last_scrub_stamp":"2024-04-25T00:46:21.966189+0000","last_deep_scrub":"22'24","last_deep_scrub_stamp":"2024-04-25T00:46:21.966189+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.966189+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:14.944131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"22'50","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.175936+0000","last_change":"2024-04-25T00:46:27.176072+0000","last_active":"2024-04-25T00:46:27.175936+0000","last_peered":"2024-04-25T00:46:27.175936+0000","last_clean":"2024-04-25T00:46:27.175936+0000","last_became_active":"2024-04-25T00:42:58.699714+0000","last_became_peered":"2024-04-25T00:42:58.699714+0000","last_unstale":"2024-04-25T00:46:27.175936+0000","last_undegraded":"2024-04-25T00:46:27.175936+0000","last_fullsized":"2024-04-25T00:46:27.175936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'50","last_scrub_stamp":"2024-04-25T00:46:27.175629+0000","last_deep_scrub":"22'50","last_deep_scrub_stamp":"2024-04-25T00:46:27.175629+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.175629+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:35.036137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'57","reported_seq":85,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.781643+0000","last_change":"2024-04-25T00:46:22.781828+0000","last_active":"2024-04-25T00:46:22.781643+0000","last_peered":"2024-04-25T00:46:22.781643+0000","last_clean":"2024-04-25T00:46:22.781643+0000","last_became_active":"2024-04-25T00:42:57.682052+0000","last_became_peered":"2024-04-25T00:42:57.682052+0000","last_unstale":"2024-04-25T00:46:22.781643+0000","last_undegraded":"2024-04-25T00:46:22.781643+0000","last_fullsized":"2024-04-25T00:46:22.781643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'57","last_scrub_stamp":"2024-04-25T00:46:22.781413+0000","last_deep_scrub":"22'57","last_deep_scrub_stamp":"2024-04-25T00:46:22.781413+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.781413+0000","objects_scrubbed":2,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:43.460204+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'49","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:23.526727+0000","last_change":"2024-04-25T00:46:23.526787+0000","last_active":"2024-04-25T00:46:23.526727+0000","last_peered":"2024-04-25T00:46:23.526727+0000","last_clean":"2024-04-25T00:46:23.526727+0000","last_became_active":"2024-04-25T00:42:58.676037+0000","last_became_peered":"2024-04-25T00:42:58.676037+0000","last_unstale":"2024-04-25T00:46:23.526727+0000","last_undegraded":"2024-04-25T00:46:23.526727+0000","last_fullsized":"2024-04-25T00:46:23.526727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'49","last_scrub_stamp":"2024-04-25T00:46:23.526536+0000","last_deep_scrub":"22'49","last_deep_scrub_stamp":"2024-04-25T00:46:23.526536+0000","last_clean_scrub_stamp":"2024-04-25T00:46:23.526536+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:39.876152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'930","reported_seq":958,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:19.940739+0000","last_change":"2024-04-25T00:46:19.940823+0000","last_active":"2024-04-25T00:46:19.940739+0000","last_peered":"2024-04-25T00:46:19.940739+0000","last_clean":"2024-04-25T00:46:19.940739+0000","last_became_active":"2024-04-25T00:42:57.650372+0000","last_became_peered":"2024-04-25T00:42:57.650372+0000","last_unstale":"2024-04-25T00:46:19.940739+0000","last_undegraded":"2024-04-25T00:46:19.940739+0000","last_fullsized":"2024-04-25T00:46:19.940739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'930","last_scrub_stamp":"2024-04-25T00:46:19.940488+0000","last_deep_scrub":"22'930","last_deep_scrub_stamp":"2024-04-25T00:46:19.940488+0000","last_clean_scrub_stamp":"2024-04-25T00:46:19.940488+0000","objects_scrubbed":4,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:56.337520+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":908,"num_write_kb":4392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"22'43","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:35.632116+0000","last_change":"2024-04-25T00:46:35.632177+0000","last_active":"2024-04-25T00:46:35.632116+0000","last_peered":"2024-04-25T00:46:35.632116+0000","last_clean":"2024-04-25T00:46:35.632116+0000","last_became_active":"2024-04-25T00:42:58.707582+0000","last_became_peered":"2024-04-25T00:42:58.707582+0000","last_unstale":"2024-04-25T00:46:35.632116+0000","last_undegraded":"2024-04-25T00:46:35.632116+0000","last_fullsized":"2024-04-25T00:46:35.632116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'43","last_scrub_stamp":"2024-04-25T00:46:35.631913+0000","last_deep_scrub":"22'43","last_deep_scrub_stamp":"2024-04-25T00:46:35.631913+0000","last_clean_scrub_stamp":"2024-04-25T00:46:35.631913+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:00.576317+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'19","reported_seq":47,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:17.363987+0000","last_change":"2024-04-25T00:46:17.364039+0000","last_active":"2024-04-25T00:46:17.363987+0000","last_peered":"2024-04-25T00:46:17.363987+0000","last_clean":"2024-04-25T00:46:17.363987+0000","last_became_active":"2024-04-25T00:42:57.657886+0000","last_became_peered":"2024-04-25T00:42:57.657886+0000","last_unstale":"2024-04-25T00:46:17.363987+0000","last_undegraded":"2024-04-25T00:46:17.363987+0000","last_fullsized":"2024-04-25T00:46:17.363987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'19","last_scrub_stamp":"2024-04-25T00:46:17.363855+0000","last_deep_scrub":"22'19","last_deep_scrub_stamp":"2024-04-25T00:46:17.363855+0000","last_clean_scrub_stamp":"2024-04-25T00:46:17.363855+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:53.760887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"22'43","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.220952+0000","last_change":"2024-04-25T00:46:25.221085+0000","last_active":"2024-04-25T00:46:25.220952+0000","last_peered":"2024-04-25T00:46:25.220952+0000","last_clean":"2024-04-25T00:46:25.220952+0000","last_became_active":"2024-04-25T00:42:58.687769+0000","last_became_peered":"2024-04-25T00:42:58.687769+0000","last_unstale":"2024-04-25T00:46:25.220952+0000","last_undegraded":"2024-04-25T00:46:25.220952+0000","last_fullsized":"2024-04-25T00:46:25.220952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'43","last_scrub_stamp":"2024-04-25T00:46:25.220712+0000","last_deep_scrub":"22'43","last_deep_scrub_stamp":"2024-04-25T00:46:25.220712+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.220712+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:59.169398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'35","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.019837+0000","last_change":"2024-04-25T00:46:24.019903+0000","last_active":"2024-04-25T00:46:24.019837+0000","last_peered":"2024-04-25T00:46:24.019837+0000","last_clean":"2024-04-25T00:46:24.019837+0000","last_became_active":"2024-04-25T00:42:57.673520+0000","last_became_peered":"2024-04-25T00:42:57.673520+0000","last_unstale":"2024-04-25T00:46:24.019837+0000","last_undegraded":"2024-04-25T00:46:24.019837+0000","last_fullsized":"2024-04-25T00:46:24.019837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'35","last_scrub_stamp":"2024-04-25T00:46:24.019641+0000","last_deep_scrub":"22'35","last_deep_scrub_stamp":"2024-04-25T00:46:24.019641+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.019641+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:27.472336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'63","reported_seq":89,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.443380+0000","last_change":"2024-04-25T00:46:20.443487+0000","last_active":"2024-04-25T00:46:20.443380+0000","last_peered":"2024-04-25T00:46:20.443380+0000","last_clean":"2024-04-25T00:46:20.443380+0000","last_became_active":"2024-04-25T00:42:58.685853+0000","last_became_peered":"2024-04-25T00:42:58.685853+0000","last_unstale":"2024-04-25T00:46:20.443380+0000","last_undegraded":"2024-04-25T00:46:20.443380+0000","last_fullsized":"2024-04-25T00:46:20.443380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'63","last_scrub_stamp":"2024-04-25T00:46:20.443189+0000","last_deep_scrub":"22'63","last_deep_scrub_stamp":"2024-04-25T00:46:20.443189+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.443189+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:59.696426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'30","reported_seq":58,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.964570+0000","last_change":"2024-04-25T00:46:21.964679+0000","last_active":"2024-04-25T00:46:21.964570+0000","last_peered":"2024-04-25T00:46:21.964570+0000","last_clean":"2024-04-25T00:46:21.964570+0000","last_became_active":"2024-04-25T00:42:57.662013+0000","last_became_peered":"2024-04-25T00:42:57.662013+0000","last_unstale":"2024-04-25T00:46:21.964570+0000","last_undegraded":"2024-04-25T00:46:21.964570+0000","last_fullsized":"2024-04-25T00:46:21.964570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'30","last_scrub_stamp":"2024-04-25T00:46:21.964277+0000","last_deep_scrub":"22'30","last_deep_scrub_stamp":"2024-04-25T00:46:21.964277+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.964277+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:21.651425+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"22'49","reported_seq":75,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.991324+0000","last_change":"2024-04-25T00:46:27.991383+0000","last_active":"2024-04-25T00:46:27.991324+0000","last_peered":"2024-04-25T00:46:27.991324+0000","last_clean":"2024-04-25T00:46:27.991324+0000","last_became_active":"2024-04-25T00:42:58.680580+0000","last_became_peered":"2024-04-25T00:42:58.680580+0000","last_unstale":"2024-04-25T00:46:27.991324+0000","last_undegraded":"2024-04-25T00:46:27.991324+0000","last_fullsized":"2024-04-25T00:46:27.991324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'49","last_scrub_stamp":"2024-04-25T00:46:27.991115+0000","last_deep_scrub":"22'49","last_deep_scrub_stamp":"2024-04-25T00:46:27.991115+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.991115+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:01.939802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'37","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:18.283784+0000","last_change":"2024-04-25T00:46:18.283856+0000","last_active":"2024-04-25T00:46:18.283784+0000","last_peered":"2024-04-25T00:46:18.283784+0000","last_clean":"2024-04-25T00:46:18.283784+0000","last_became_active":"2024-04-25T00:42:57.653513+0000","last_became_peered":"2024-04-25T00:42:57.653513+0000","last_unstale":"2024-04-25T00:46:18.283784+0000","last_undegraded":"2024-04-25T00:46:18.283784+0000","last_fullsized":"2024-04-25T00:46:18.283784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'37","last_scrub_stamp":"2024-04-25T00:46:18.283597+0000","last_deep_scrub":"22'37","last_deep_scrub_stamp":"2024-04-25T00:46:18.283597+0000","last_clean_scrub_stamp":"2024-04-25T00:46:18.283597+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:43.008603+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"22'47","reported_seq":73,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:28.541870+0000","last_change":"2024-04-25T00:46:28.541931+0000","last_active":"2024-04-25T00:46:28.541870+0000","last_peered":"2024-04-25T00:46:28.541870+0000","last_clean":"2024-04-25T00:46:28.541870+0000","last_became_active":"2024-04-25T00:42:58.685279+0000","last_became_peered":"2024-04-25T00:42:58.685279+0000","last_unstale":"2024-04-25T00:46:28.541870+0000","last_undegraded":"2024-04-25T00:46:28.541870+0000","last_fullsized":"2024-04-25T00:46:28.541870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'47","last_scrub_stamp":"2024-04-25T00:46:28.541675+0000","last_deep_scrub":"22'47","last_deep_scrub_stamp":"2024-04-25T00:46:28.541675+0000","last_clean_scrub_stamp":"2024-04-25T00:46:28.541675+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:10.477112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'24","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:19.553026+0000","last_change":"2024-04-25T00:46:19.553095+0000","last_active":"2024-04-25T00:46:19.553026+0000","last_peered":"2024-04-25T00:46:19.553026+0000","last_clean":"2024-04-25T00:46:19.553026+0000","last_became_active":"2024-04-25T00:42:57.656362+0000","last_became_peered":"2024-04-25T00:42:57.656362+0000","last_unstale":"2024-04-25T00:46:19.553026+0000","last_undegraded":"2024-04-25T00:46:19.553026+0000","last_fullsized":"2024-04-25T00:46:19.553026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'24","last_scrub_stamp":"2024-04-25T00:46:19.552824+0000","last_deep_scrub":"22'24","last_deep_scrub_stamp":"2024-04-25T00:46:19.552824+0000","last_clean_scrub_stamp":"2024-04-25T00:46:19.552824+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:44.277830+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"22'39","reported_seq":65,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:32.646135+0000","last_change":"2024-04-25T00:46:32.646305+0000","last_active":"2024-04-25T00:46:32.646135+0000","last_peered":"2024-04-25T00:46:32.646135+0000","last_clean":"2024-04-25T00:46:32.646135+0000","last_became_active":"2024-04-25T00:42:58.691339+0000","last_became_peered":"2024-04-25T00:42:58.691339+0000","last_unstale":"2024-04-25T00:46:32.646135+0000","last_undegraded":"2024-04-25T00:46:32.646135+0000","last_fullsized":"2024-04-25T00:46:32.646135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'39","last_scrub_stamp":"2024-04-25T00:46:32.645896+0000","last_deep_scrub":"22'39","last_deep_scrub_stamp":"2024-04-25T00:46:32.645896+0000","last_clean_scrub_stamp":"2024-04-25T00:46:32.645896+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:39.494749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'21","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:19.324300+0000","last_change":"2024-04-25T00:46:19.324323+0000","last_active":"2024-04-25T00:46:19.324300+0000","last_peered":"2024-04-25T00:46:19.324300+0000","last_clean":"2024-04-25T00:46:19.324300+0000","last_became_active":"2024-04-25T00:42:57.658097+0000","last_became_peered":"2024-04-25T00:42:57.658097+0000","last_unstale":"2024-04-25T00:46:19.324300+0000","last_undegraded":"2024-04-25T00:46:19.324300+0000","last_fullsized":"2024-04-25T00:46:19.324300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'21","last_scrub_stamp":"2024-04-25T00:46:19.324221+0000","last_deep_scrub":"22'21","last_deep_scrub_stamp":"2024-04-25T00:46:19.324221+0000","last_clean_scrub_stamp":"2024-04-25T00:46:19.324221+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:45.556975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"22'47","reported_seq":73,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:33.650623+0000","last_change":"2024-04-25T00:46:33.650774+0000","last_active":"2024-04-25T00:46:33.650623+0000","last_peered":"2024-04-25T00:46:33.650623+0000","last_clean":"2024-04-25T00:46:33.650623+0000","last_became_active":"2024-04-25T00:42:58.695772+0000","last_became_peered":"2024-04-25T00:42:58.695772+0000","last_unstale":"2024-04-25T00:46:33.650623+0000","last_undegraded":"2024-04-25T00:46:33.650623+0000","last_fullsized":"2024-04-25T00:46:33.650623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'47","last_scrub_stamp":"2024-04-25T00:46:33.650378+0000","last_deep_scrub":"22'47","last_deep_scrub_stamp":"2024-04-25T00:46:33.650378+0000","last_clean_scrub_stamp":"2024-04-25T00:46:33.650378+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:54.090866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'25","reported_seq":53,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:19.754134+0000","last_change":"2024-04-25T00:46:19.754158+0000","last_active":"2024-04-25T00:46:19.754134+0000","last_peered":"2024-04-25T00:46:19.754134+0000","last_clean":"2024-04-25T00:46:19.754134+0000","last_became_active":"2024-04-25T00:42:57.656453+0000","last_became_peered":"2024-04-25T00:42:57.656453+0000","last_unstale":"2024-04-25T00:46:19.754134+0000","last_undegraded":"2024-04-25T00:46:19.754134+0000","last_fullsized":"2024-04-25T00:46:19.754134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'25","last_scrub_stamp":"2024-04-25T00:46:19.754067+0000","last_deep_scrub":"22'25","last_deep_scrub_stamp":"2024-04-25T00:46:19.754067+0000","last_clean_scrub_stamp":"2024-04-25T00:46:19.754067+0000","objects_scrubbed":1,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:38.940232+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"22'43","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:34.990654+0000","last_change":"2024-04-25T00:46:34.990717+0000","last_active":"2024-04-25T00:46:34.990654+0000","last_peered":"2024-04-25T00:46:34.990654+0000","last_clean":"2024-04-25T00:46:34.990654+0000","last_became_active":"2024-04-25T00:43:39.827957+0000","last_became_peered":"2024-04-25T00:43:39.827957+0000","last_unstale":"2024-04-25T00:46:34.990654+0000","last_undegraded":"2024-04-25T00:46:34.990654+0000","last_fullsized":"2024-04-25T00:46:34.990654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'43","last_scrub_stamp":"2024-04-25T00:46:34.990462+0000","last_deep_scrub":"22'43","last_deep_scrub_stamp":"2024-04-25T00:46:34.990462+0000","last_clean_scrub_stamp":"2024-04-25T00:46:34.990462+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:15.700945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'29","reported_seq":57,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.280954+0000","last_change":"2024-04-25T00:46:21.281016+0000","last_active":"2024-04-25T00:46:21.280954+0000","last_peered":"2024-04-25T00:46:21.280954+0000","last_clean":"2024-04-25T00:46:21.280954+0000","last_became_active":"2024-04-25T00:42:57.673844+0000","last_became_peered":"2024-04-25T00:42:57.673844+0000","last_unstale":"2024-04-25T00:46:21.280954+0000","last_undegraded":"2024-04-25T00:46:21.280954+0000","last_fullsized":"2024-04-25T00:46:21.280954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'29","last_scrub_stamp":"2024-04-25T00:46:21.280752+0000","last_deep_scrub":"22'29","last_deep_scrub_stamp":"2024-04-25T00:46:21.280752+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.280752+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:06.372321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'46","reported_seq":66,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.303300+0000","last_change":"2024-04-25T00:46:25.303363+0000","last_active":"2024-04-25T00:46:25.303300+0000","last_peered":"2024-04-25T00:46:25.303300+0000","last_clean":"2024-04-25T00:46:25.303300+0000","last_became_active":"2024-04-25T00:43:39.821411+0000","last_became_peered":"2024-04-25T00:43:39.821411+0000","last_unstale":"2024-04-25T00:46:25.303300+0000","last_undegraded":"2024-04-25T00:46:25.303300+0000","last_fullsized":"2024-04-25T00:46:25.303300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'46","last_scrub_stamp":"2024-04-25T00:46:25.303102+0000","last_deep_scrub":"22'46","last_deep_scrub_stamp":"2024-04-25T00:46:25.303102+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.303102+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:54.633321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":186,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.466434+0000","last_change":"2024-04-25T00:46:21.466506+0000","last_active":"2024-04-25T00:46:21.466434+0000","last_peered":"2024-04-25T00:46:21.466434+0000","last_clean":"2024-04-25T00:46:21.466434+0000","last_became_active":"2024-04-25T00:42:48.006156+0000","last_became_peered":"2024-04-25T00:42:48.006156+0000","last_unstale":"2024-04-25T00:46:21.466434+0000","last_undegraded":"2024-04-25T00:46:21.466434+0000","last_fullsized":"2024-04-25T00:46:21.466434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2024-04-25T00:46:21.466248+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2024-04-25T00:46:21.466248+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.466248+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:45:49.356090+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'20","reported_seq":48,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.996520+0000","last_change":"2024-04-25T00:46:20.996677+0000","last_active":"2024-04-25T00:46:20.996520+0000","last_peered":"2024-04-25T00:46:20.996520+0000","last_clean":"2024-04-25T00:46:20.996520+0000","last_became_active":"2024-04-25T00:42:57.668608+0000","last_became_peered":"2024-04-25T00:42:57.668608+0000","last_unstale":"2024-04-25T00:46:20.996520+0000","last_undegraded":"2024-04-25T00:46:20.996520+0000","last_fullsized":"2024-04-25T00:46:20.996520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'20","last_scrub_stamp":"2024-04-25T00:46:20.996265+0000","last_deep_scrub":"22'20","last_deep_scrub_stamp":"2024-04-25T00:46:20.996265+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.996265+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:07:41.675056+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"22'57","reported_seq":83,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:30.107985+0000","last_change":"2024-04-25T00:46:30.108044+0000","last_active":"2024-04-25T00:46:30.107985+0000","last_peered":"2024-04-25T00:46:30.107985+0000","last_clean":"2024-04-25T00:46:30.107985+0000","last_became_active":"2024-04-25T00:42:58.697431+0000","last_became_peered":"2024-04-25T00:42:58.697431+0000","last_unstale":"2024-04-25T00:46:30.107985+0000","last_undegraded":"2024-04-25T00:46:30.107985+0000","last_fullsized":"2024-04-25T00:46:30.107985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'57","last_scrub_stamp":"2024-04-25T00:46:30.107784+0000","last_deep_scrub":"22'57","last_deep_scrub_stamp":"2024-04-25T00:46:30.107784+0000","last_clean_scrub_stamp":"2024-04-25T00:46:30.107784+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:12.043221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'21","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:23.483989+0000","last_change":"2024-04-25T00:46:23.484053+0000","last_active":"2024-04-25T00:46:23.483989+0000","last_peered":"2024-04-25T00:46:23.483989+0000","last_clean":"2024-04-25T00:46:23.483989+0000","last_became_active":"2024-04-25T00:42:57.680788+0000","last_became_peered":"2024-04-25T00:42:57.680788+0000","last_unstale":"2024-04-25T00:46:23.483989+0000","last_undegraded":"2024-04-25T00:46:23.483989+0000","last_fullsized":"2024-04-25T00:46:23.483989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'21","last_scrub_stamp":"2024-04-25T00:46:23.483796+0000","last_deep_scrub":"22'21","last_deep_scrub_stamp":"2024-04-25T00:46:23.483796+0000","last_clean_scrub_stamp":"2024-04-25T00:46:23.483796+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:36:26.936491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"22'43","reported_seq":69,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:19.288454+0000","last_change":"2024-04-25T00:46:19.288566+0000","last_active":"2024-04-25T00:46:19.288454+0000","last_peered":"2024-04-25T00:46:19.288454+0000","last_clean":"2024-04-25T00:46:19.288454+0000","last_became_active":"2024-04-25T00:42:58.656714+0000","last_became_peered":"2024-04-25T00:42:58.656714+0000","last_unstale":"2024-04-25T00:46:19.288454+0000","last_undegraded":"2024-04-25T00:46:19.288454+0000","last_fullsized":"2024-04-25T00:46:19.288454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'43","last_scrub_stamp":"2024-04-25T00:46:19.288220+0000","last_deep_scrub":"22'43","last_deep_scrub_stamp":"2024-04-25T00:46:19.288220+0000","last_clean_scrub_stamp":"2024-04-25T00:46:19.288220+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:18.975368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'94","reported_seq":187,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.287854+0000","last_change":"2024-04-25T00:46:20.287893+0000","last_active":"2024-04-25T00:46:20.287854+0000","last_peered":"2024-04-25T00:46:20.287854+0000","last_clean":"2024-04-25T00:46:20.287854+0000","last_became_active":"2024-04-25T00:42:57.659713+0000","last_became_peered":"2024-04-25T00:42:57.659713+0000","last_unstale":"2024-04-25T00:46:20.287854+0000","last_undegraded":"2024-04-25T00:46:20.287854+0000","last_fullsized":"2024-04-25T00:46:20.287854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'94","last_scrub_stamp":"2024-04-25T00:46:20.287733+0000","last_deep_scrub":"22'94","last_deep_scrub_stamp":"2024-04-25T00:46:20.287733+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.287733+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:42.132822+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357237,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":65,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'45","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:28.711084+0000","last_change":"2024-04-25T00:46:28.711147+0000","last_active":"2024-04-25T00:46:28.711084+0000","last_peered":"2024-04-25T00:46:28.711084+0000","last_clean":"2024-04-25T00:46:28.711084+0000","last_became_active":"2024-04-25T00:42:58.693733+0000","last_became_peered":"2024-04-25T00:42:58.693733+0000","last_unstale":"2024-04-25T00:46:28.711084+0000","last_undegraded":"2024-04-25T00:46:28.711084+0000","last_fullsized":"2024-04-25T00:46:28.711084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'45","last_scrub_stamp":"2024-04-25T00:46:28.710869+0000","last_deep_scrub":"22'45","last_deep_scrub_stamp":"2024-04-25T00:46:28.710869+0000","last_clean_scrub_stamp":"2024-04-25T00:46:28.710869+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:26.377091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'24","reported_seq":52,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.038617+0000","last_change":"2024-04-25T00:46:20.038680+0000","last_active":"2024-04-25T00:46:20.038617+0000","last_peered":"2024-04-25T00:46:20.038617+0000","last_clean":"2024-04-25T00:46:20.038617+0000","last_became_active":"2024-04-25T00:42:57.666472+0000","last_became_peered":"2024-04-25T00:42:57.666472+0000","last_unstale":"2024-04-25T00:46:20.038617+0000","last_undegraded":"2024-04-25T00:46:20.038617+0000","last_fullsized":"2024-04-25T00:46:20.038617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'24","last_scrub_stamp":"2024-04-25T00:46:20.038432+0000","last_deep_scrub":"22'24","last_deep_scrub_stamp":"2024-04-25T00:46:20.038432+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.038432+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:42:59.291668+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.589142+0000","last_change":"2024-04-25T00:46:21.589200+0000","last_active":"2024-04-25T00:46:21.589142+0000","last_peered":"2024-04-25T00:46:21.589142+0000","last_clean":"2024-04-25T00:46:21.589142+0000","last_became_active":"2024-04-25T00:42:58.664525+0000","last_became_peered":"2024-04-25T00:42:58.664525+0000","last_unstale":"2024-04-25T00:46:21.589142+0000","last_undegraded":"2024-04-25T00:46:21.589142+0000","last_fullsized":"2024-04-25T00:46:21.589142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:21.588944+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:21.588944+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.588944+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:10.227685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'22","reported_seq":50,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:18.751756+0000","last_change":"2024-04-25T00:46:18.751785+0000","last_active":"2024-04-25T00:46:18.751756+0000","last_peered":"2024-04-25T00:46:18.751756+0000","last_clean":"2024-04-25T00:46:18.751756+0000","last_became_active":"2024-04-25T00:42:57.656530+0000","last_became_peered":"2024-04-25T00:42:57.656530+0000","last_unstale":"2024-04-25T00:46:18.751756+0000","last_undegraded":"2024-04-25T00:46:18.751756+0000","last_fullsized":"2024-04-25T00:46:18.751756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'22","last_scrub_stamp":"2024-04-25T00:46:18.751641+0000","last_deep_scrub":"22'22","last_deep_scrub_stamp":"2024-04-25T00:46:18.751641+0000","last_clean_scrub_stamp":"2024-04-25T00:46:18.751641+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:10.687066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'46","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.038130+0000","last_change":"2024-04-25T00:46:26.038202+0000","last_active":"2024-04-25T00:46:26.038130+0000","last_peered":"2024-04-25T00:46:26.038130+0000","last_clean":"2024-04-25T00:46:26.038130+0000","last_became_active":"2024-04-25T00:42:58.690387+0000","last_became_peered":"2024-04-25T00:42:58.690387+0000","last_unstale":"2024-04-25T00:46:26.038130+0000","last_undegraded":"2024-04-25T00:46:26.038130+0000","last_fullsized":"2024-04-25T00:46:26.038130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'46","last_scrub_stamp":"2024-04-25T00:46:26.037926+0000","last_deep_scrub":"22'46","last_deep_scrub_stamp":"2024-04-25T00:46:26.037926+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.037926+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:24.711598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'28","reported_seq":56,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.531499+0000","last_change":"2024-04-25T00:46:24.531566+0000","last_active":"2024-04-25T00:46:24.531499+0000","last_peered":"2024-04-25T00:46:24.531499+0000","last_clean":"2024-04-25T00:46:24.531499+0000","last_became_active":"2024-04-25T00:42:57.685162+0000","last_became_peered":"2024-04-25T00:42:57.685162+0000","last_unstale":"2024-04-25T00:46:24.531499+0000","last_undegraded":"2024-04-25T00:46:24.531499+0000","last_fullsized":"2024-04-25T00:46:24.531499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'28","last_scrub_stamp":"2024-04-25T00:46:24.531307+0000","last_deep_scrub":"22'28","last_deep_scrub_stamp":"2024-04-25T00:46:24.531307+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.531307+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:21.328261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'45","reported_seq":71,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:23.745585+0000","last_change":"2024-04-25T00:46:23.745643+0000","last_active":"2024-04-25T00:46:23.745585+0000","last_peered":"2024-04-25T00:46:23.745585+0000","last_clean":"2024-04-25T00:46:23.745585+0000","last_became_active":"2024-04-25T00:42:58.675526+0000","last_became_peered":"2024-04-25T00:42:58.675526+0000","last_unstale":"2024-04-25T00:46:23.745585+0000","last_undegraded":"2024-04-25T00:46:23.745585+0000","last_fullsized":"2024-04-25T00:46:23.745585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'45","last_scrub_stamp":"2024-04-25T00:46:23.745392+0000","last_deep_scrub":"22'45","last_deep_scrub_stamp":"2024-04-25T00:46:23.745392+0000","last_clean_scrub_stamp":"2024-04-25T00:46:23.745392+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:16.723335+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'94","reported_seq":122,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:17.746669+0000","last_change":"2024-04-25T00:46:17.746742+0000","last_active":"2024-04-25T00:46:17.746669+0000","last_peered":"2024-04-25T00:46:17.746669+0000","last_clean":"2024-04-25T00:46:17.746669+0000","last_became_active":"2024-04-25T00:42:57.651505+0000","last_became_peered":"2024-04-25T00:42:57.651505+0000","last_unstale":"2024-04-25T00:46:17.746669+0000","last_undegraded":"2024-04-25T00:46:17.746669+0000","last_fullsized":"2024-04-25T00:46:17.746669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'94","last_scrub_stamp":"2024-04-25T00:46:17.746450+0000","last_deep_scrub":"22'94","last_deep_scrub_stamp":"2024-04-25T00:46:17.746450+0000","last_clean_scrub_stamp":"2024-04-25T00:46:17.746450+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:55.720904+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'51","reported_seq":77,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.090701+0000","last_change":"2024-04-25T00:46:25.090765+0000","last_active":"2024-04-25T00:46:25.090701+0000","last_peered":"2024-04-25T00:46:25.090701+0000","last_clean":"2024-04-25T00:46:25.090701+0000","last_became_active":"2024-04-25T00:42:58.692509+0000","last_became_peered":"2024-04-25T00:42:58.692509+0000","last_unstale":"2024-04-25T00:46:25.090701+0000","last_undegraded":"2024-04-25T00:46:25.090701+0000","last_fullsized":"2024-04-25T00:46:25.090701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'51","last_scrub_stamp":"2024-04-25T00:46:25.090506+0000","last_deep_scrub":"22'51","last_deep_scrub_stamp":"2024-04-25T00:46:25.090506+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.090506+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:43:39.882021+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'27","reported_seq":55,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.979328+0000","last_change":"2024-04-25T00:46:22.979476+0000","last_active":"2024-04-25T00:46:22.979328+0000","last_peered":"2024-04-25T00:46:22.979328+0000","last_clean":"2024-04-25T00:46:22.979328+0000","last_became_active":"2024-04-25T00:42:57.679375+0000","last_became_peered":"2024-04-25T00:42:57.679375+0000","last_unstale":"2024-04-25T00:46:22.979328+0000","last_undegraded":"2024-04-25T00:46:22.979328+0000","last_fullsized":"2024-04-25T00:46:22.979328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'27","last_scrub_stamp":"2024-04-25T00:46:22.979076+0000","last_deep_scrub":"22'27","last_deep_scrub_stamp":"2024-04-25T00:46:22.979076+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.979076+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:03.907840+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'58","reported_seq":84,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.222920+0000","last_change":"2024-04-25T00:46:26.222989+0000","last_active":"2024-04-25T00:46:26.222920+0000","last_peered":"2024-04-25T00:46:26.222920+0000","last_clean":"2024-04-25T00:46:26.222920+0000","last_became_active":"2024-04-25T00:42:58.699523+0000","last_became_peered":"2024-04-25T00:42:58.699523+0000","last_unstale":"2024-04-25T00:46:26.222920+0000","last_undegraded":"2024-04-25T00:46:26.222920+0000","last_fullsized":"2024-04-25T00:46:26.222920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'58","last_scrub_stamp":"2024-04-25T00:46:26.222689+0000","last_deep_scrub":"22'58","last_deep_scrub_stamp":"2024-04-25T00:46:26.222689+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.222689+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:07.850442+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'21","reported_seq":49,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.527205+0000","last_change":"2024-04-25T00:46:25.527334+0000","last_active":"2024-04-25T00:46:25.527205+0000","last_peered":"2024-04-25T00:46:25.527205+0000","last_clean":"2024-04-25T00:46:25.527205+0000","last_became_active":"2024-04-25T00:42:57.683324+0000","last_became_peered":"2024-04-25T00:42:57.683324+0000","last_unstale":"2024-04-25T00:46:25.527205+0000","last_undegraded":"2024-04-25T00:46:25.527205+0000","last_fullsized":"2024-04-25T00:46:25.527205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'21","last_scrub_stamp":"2024-04-25T00:46:25.526929+0000","last_deep_scrub":"22'21","last_deep_scrub_stamp":"2024-04-25T00:46:25.526929+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.526929+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:24.200601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"22'48","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.586168+0000","last_change":"2024-04-25T00:46:20.586231+0000","last_active":"2024-04-25T00:46:20.586168+0000","last_peered":"2024-04-25T00:46:20.586168+0000","last_clean":"2024-04-25T00:46:20.586168+0000","last_became_active":"2024-04-25T00:42:58.659384+0000","last_became_peered":"2024-04-25T00:42:58.659384+0000","last_unstale":"2024-04-25T00:46:20.586168+0000","last_undegraded":"2024-04-25T00:46:20.586168+0000","last_fullsized":"2024-04-25T00:46:20.586168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'48","last_scrub_stamp":"2024-04-25T00:46:20.585969+0000","last_deep_scrub":"22'48","last_deep_scrub_stamp":"2024-04-25T00:46:20.585969+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.585969+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:46.818723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'28","reported_seq":56,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:21.729514+0000","last_change":"2024-04-25T00:46:21.729579+0000","last_active":"2024-04-25T00:46:21.729514+0000","last_peered":"2024-04-25T00:46:21.729514+0000","last_clean":"2024-04-25T00:46:21.729514+0000","last_became_active":"2024-04-25T00:42:57.676352+0000","last_became_peered":"2024-04-25T00:42:57.676352+0000","last_unstale":"2024-04-25T00:46:21.729514+0000","last_undegraded":"2024-04-25T00:46:21.729514+0000","last_fullsized":"2024-04-25T00:46:21.729514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'28","last_scrub_stamp":"2024-04-25T00:46:21.729318+0000","last_deep_scrub":"22'28","last_deep_scrub_stamp":"2024-04-25T00:46:21.729318+0000","last_clean_scrub_stamp":"2024-04-25T00:46:21.729318+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:00.982554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"22'48","reported_seq":74,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.117171+0000","last_change":"2024-04-25T00:46:26.117235+0000","last_active":"2024-04-25T00:46:26.117171+0000","last_peered":"2024-04-25T00:46:26.117171+0000","last_clean":"2024-04-25T00:46:26.117171+0000","last_became_active":"2024-04-25T00:42:58.693144+0000","last_became_peered":"2024-04-25T00:42:58.693144+0000","last_unstale":"2024-04-25T00:46:26.117171+0000","last_undegraded":"2024-04-25T00:46:26.117171+0000","last_fullsized":"2024-04-25T00:46:26.117171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'48","last_scrub_stamp":"2024-04-25T00:46:26.116971+0000","last_deep_scrub":"22'48","last_deep_scrub_stamp":"2024-04-25T00:46:26.116971+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.116971+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:22.913924+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'31","reported_seq":59,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.778348+0000","last_change":"2024-04-25T00:46:20.778413+0000","last_active":"2024-04-25T00:46:20.778348+0000","last_peered":"2024-04-25T00:46:20.778348+0000","last_clean":"2024-04-25T00:46:20.778348+0000","last_became_active":"2024-04-25T00:42:57.658103+0000","last_became_peered":"2024-04-25T00:42:57.658103+0000","last_unstale":"2024-04-25T00:46:20.778348+0000","last_undegraded":"2024-04-25T00:46:20.778348+0000","last_fullsized":"2024-04-25T00:46:20.778348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'31","last_scrub_stamp":"2024-04-25T00:46:20.778149+0000","last_deep_scrub":"22'31","last_deep_scrub_stamp":"2024-04-25T00:46:20.778149+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.778149+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:50.108369+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"22'55","reported_seq":81,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.096492+0000","last_change":"2024-04-25T00:46:27.096555+0000","last_active":"2024-04-25T00:46:27.096492+0000","last_peered":"2024-04-25T00:46:27.096492+0000","last_clean":"2024-04-25T00:46:27.096492+0000","last_became_active":"2024-04-25T00:42:58.696419+0000","last_became_peered":"2024-04-25T00:42:58.696419+0000","last_unstale":"2024-04-25T00:46:27.096492+0000","last_undegraded":"2024-04-25T00:46:27.096492+0000","last_fullsized":"2024-04-25T00:46:27.096492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'55","last_scrub_stamp":"2024-04-25T00:46:27.096287+0000","last_deep_scrub":"22'55","last_deep_scrub_stamp":"2024-04-25T00:46:27.096287+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.096287+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:25.769959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'510","reported_seq":538,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:22.476131+0000","last_change":"2024-04-25T00:46:22.476325+0000","last_active":"2024-04-25T00:46:22.476131+0000","last_peered":"2024-04-25T00:46:22.476131+0000","last_clean":"2024-04-25T00:46:22.476131+0000","last_became_active":"2024-04-25T00:42:57.680658+0000","last_became_peered":"2024-04-25T00:42:57.680658+0000","last_unstale":"2024-04-25T00:46:22.476131+0000","last_undegraded":"2024-04-25T00:46:22.476131+0000","last_fullsized":"2024-04-25T00:46:22.476131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'510","last_scrub_stamp":"2024-04-25T00:46:22.475895+0000","last_deep_scrub":"22'510","last_deep_scrub_stamp":"2024-04-25T00:46:22.475895+0000","last_clean_scrub_stamp":"2024-04-25T00:46:22.475895+0000","objects_scrubbed":2,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:06.709816+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"22'42","reported_seq":68,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:25.008667+0000","last_change":"2024-04-25T00:46:25.008828+0000","last_active":"2024-04-25T00:46:25.008667+0000","last_peered":"2024-04-25T00:46:25.008667+0000","last_clean":"2024-04-25T00:46:25.008667+0000","last_became_active":"2024-04-25T00:42:58.669580+0000","last_became_peered":"2024-04-25T00:42:58.669580+0000","last_unstale":"2024-04-25T00:46:25.008667+0000","last_undegraded":"2024-04-25T00:46:25.008667+0000","last_fullsized":"2024-04-25T00:46:25.008667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'42","last_scrub_stamp":"2024-04-25T00:46:25.008399+0000","last_deep_scrub":"22'42","last_deep_scrub_stamp":"2024-04-25T00:46:25.008399+0000","last_clean_scrub_stamp":"2024-04-25T00:46:25.008399+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:17.986341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"22'58","reported_seq":84,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:30.954746+0000","last_change":"2024-04-25T00:46:30.954807+0000","last_active":"2024-04-25T00:46:30.954746+0000","last_peered":"2024-04-25T00:46:30.954746+0000","last_clean":"2024-04-25T00:46:30.954746+0000","last_became_active":"2024-04-25T00:42:58.693120+0000","last_became_peered":"2024-04-25T00:42:58.693120+0000","last_unstale":"2024-04-25T00:46:30.954746+0000","last_undegraded":"2024-04-25T00:46:30.954746+0000","last_fullsized":"2024-04-25T00:46:30.954746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'58","last_scrub_stamp":"2024-04-25T00:46:30.954553+0000","last_deep_scrub":"22'58","last_deep_scrub_stamp":"2024-04-25T00:46:30.954553+0000","last_clean_scrub_stamp":"2024-04-25T00:46:30.954553+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:08.945191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'17","reported_seq":45,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.526052+0000","last_change":"2024-04-25T00:46:27.526120+0000","last_active":"2024-04-25T00:46:27.526052+0000","last_peered":"2024-04-25T00:46:27.526052+0000","last_clean":"2024-04-25T00:46:27.526052+0000","last_became_active":"2024-04-25T00:42:57.687967+0000","last_became_peered":"2024-04-25T00:42:57.687967+0000","last_unstale":"2024-04-25T00:46:27.526052+0000","last_undegraded":"2024-04-25T00:46:27.526052+0000","last_fullsized":"2024-04-25T00:46:27.526052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'17","last_scrub_stamp":"2024-04-25T00:46:27.525843+0000","last_deep_scrub":"22'17","last_deep_scrub_stamp":"2024-04-25T00:46:27.525843+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.525843+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:57.231440+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'50","reported_seq":76,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:27.030854+0000","last_change":"2024-04-25T00:46:27.030918+0000","last_active":"2024-04-25T00:46:27.030854+0000","last_peered":"2024-04-25T00:46:27.030854+0000","last_clean":"2024-04-25T00:46:27.030854+0000","last_became_active":"2024-04-25T00:42:58.692518+0000","last_became_peered":"2024-04-25T00:42:58.692518+0000","last_unstale":"2024-04-25T00:46:27.030854+0000","last_undegraded":"2024-04-25T00:46:27.030854+0000","last_fullsized":"2024-04-25T00:46:27.030854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'50","last_scrub_stamp":"2024-04-25T00:46:27.030655+0000","last_deep_scrub":"22'50","last_deep_scrub_stamp":"2024-04-25T00:46:27.030655+0000","last_clean_scrub_stamp":"2024-04-25T00:46:27.030655+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:34.891163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'64","reported_seq":92,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:26.498909+0000","last_change":"2024-04-25T00:46:26.498987+0000","last_active":"2024-04-25T00:46:26.498909+0000","last_peered":"2024-04-25T00:46:26.498909+0000","last_clean":"2024-04-25T00:46:26.498909+0000","last_became_active":"2024-04-25T00:42:57.685924+0000","last_became_peered":"2024-04-25T00:42:57.685924+0000","last_unstale":"2024-04-25T00:46:26.498909+0000","last_undegraded":"2024-04-25T00:46:26.498909+0000","last_fullsized":"2024-04-25T00:46:26.498909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'64","last_scrub_stamp":"2024-04-25T00:46:26.498684+0000","last_deep_scrub":"22'64","last_deep_scrub_stamp":"2024-04-25T00:46:26.498684+0000","last_clean_scrub_stamp":"2024-04-25T00:46:26.498684+0000","objects_scrubbed":3,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:34.359192+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5442977,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":5338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"22'57","reported_seq":88,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:28.197200+0000","last_change":"2024-04-25T00:46:28.197264+0000","last_active":"2024-04-25T00:46:28.197200+0000","last_peered":"2024-04-25T00:46:28.197200+0000","last_clean":"2024-04-25T00:46:28.197200+0000","last_became_active":"2024-04-25T00:43:39.828089+0000","last_became_peered":"2024-04-25T00:43:39.828089+0000","last_unstale":"2024-04-25T00:46:28.197200+0000","last_undegraded":"2024-04-25T00:46:28.197200+0000","last_fullsized":"2024-04-25T00:46:28.197200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'57","last_scrub_stamp":"2024-04-25T00:46:28.197002+0000","last_deep_scrub":"22'57","last_deep_scrub_stamp":"2024-04-25T00:46:28.197002+0000","last_clean_scrub_stamp":"2024-04-25T00:46:28.197002+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:25.040870+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'35","reported_seq":63,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:20.062476+0000","last_change":"2024-04-25T00:46:20.062550+0000","last_active":"2024-04-25T00:46:20.062476+0000","last_peered":"2024-04-25T00:46:20.062476+0000","last_clean":"2024-04-25T00:46:20.062476+0000","last_became_active":"2024-04-25T00:42:57.654359+0000","last_became_peered":"2024-04-25T00:42:57.654359+0000","last_unstale":"2024-04-25T00:46:20.062476+0000","last_undegraded":"2024-04-25T00:46:20.062476+0000","last_fullsized":"2024-04-25T00:46:20.062476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'35","last_scrub_stamp":"2024-04-25T00:46:20.062284+0000","last_deep_scrub":"22'35","last_deep_scrub_stamp":"2024-04-25T00:46:20.062284+0000","last_clean_scrub_stamp":"2024-04-25T00:46:20.062284+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:21.174573+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'46","reported_seq":72,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:24.124599+0000","last_change":"2024-04-25T00:46:24.124661+0000","last_active":"2024-04-25T00:46:24.124599+0000","last_peered":"2024-04-25T00:46:24.124599+0000","last_clean":"2024-04-25T00:46:24.124599+0000","last_became_active":"2024-04-25T00:42:58.676328+0000","last_became_peered":"2024-04-25T00:42:58.676328+0000","last_unstale":"2024-04-25T00:46:24.124599+0000","last_undegraded":"2024-04-25T00:46:24.124599+0000","last_fullsized":"2024-04-25T00:46:24.124599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'46","last_scrub_stamp":"2024-04-25T00:46:24.124406+0000","last_deep_scrub":"22'46","last_deep_scrub_stamp":"2024-04-25T00:46:24.124406+0000","last_clean_scrub_stamp":"2024-04-25T00:46:24.124406+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:08.358327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'51","reported_seq":79,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T00:46:18.324649+0000","last_change":"2024-04-25T00:46:18.324730+0000","last_active":"2024-04-25T00:46:18.324649+0000","last_peered":"2024-04-25T00:46:18.324649+0000","last_clean":"2024-04-25T00:46:18.324649+0000","last_became_active":"2024-04-25T00:42:57.679443+0000","last_became_peered":"2024-04-25T00:42:57.679443+0000","last_unstale":"2024-04-25T00:46:18.324649+0000","last_undegraded":"2024-04-25T00:46:18.324649+0000","last_fullsized":"2024-04-25T00:46:18.324649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'51","last_scrub_stamp":"2024-04-25T00:46:18.324482+0000","last_deep_scrub":"22'51","last_deep_scrub_stamp":"2024-04-25T00:46:18.324482+0000","last_clean_scrub_stamp":"2024-04-25T00:46:18.324482+0000","objects_scrubbed":3,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:38.709434+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":4111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3307,"ondisk_log_size":3307,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":30967508,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":65,"num_read_kb":391,"num_write":3252,"num_write_kb":31581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61997056,"data_stored":61935016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3918,"ondisk_log_size":3918,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57908,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94313932,"statfs":{"total":96636764160,"available":96577466368,"internally_reserved":0,"allocated":18853888,"data_stored":18700145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14069760,"data_stored":13915563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":83396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39604,"kb_used_data":8428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332236,"statfs":{"total":96636764160,"available":96596209664,"internally_reserved":0,"allocated":8630272,"data_stored":8472040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52368,"kb_used_data":12872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94319472,"statfs":{"total":96636764160,"available":96583139328,"internally_reserved":0,"allocated":13180928,"data_stored":13024127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":589824,"data_stored":440543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":83466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":9224192,"data_stored":9062920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":357237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12951552,"data_stored":12940821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8388734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13840384,"data_stored":13832257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18030592,"data_stored":18026471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:46:40.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T00:46:40.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0c45c640 1 Processor -- start 2024-04-25T00:46:40.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0c45c640 1 -- start start 2024-04-25T00:46:40.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0c45c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a041428c0 0x7f0a04169140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0c45c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a040d2710 con 0x7f0a04144570 2024-04-25T00:46:40.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0c45c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a040d29c0 con 0x7f0a04141010 2024-04-25T00:46:40.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0c45c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0a040d2ca0 con 0x7f0a041428c0 2024-04-25T00:46:40.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0a9f9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a041428c0 0x7f0a04169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0a9f9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a041428c0 0x7f0a04169140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55570/0 (socket says 172.21.15.129:55570) 2024-04-25T00:46:40.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a099f7640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0a04144570 0x7f0a04143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55614/0 (socket says 172.21.15.129:55614) 2024-04-25T00:46:40.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0a9f9640 1 -- 172.21.15.129:0/2222166738 learned_addr learned my addr 172.21.15.129:0/2222166738 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:40.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0a9f9640 1 -- 172.21.15.129:0/2222166738 >> v1:172.21.15.196:6789/0 conn(0x7f0a04141010 legacy=0x7f0a040d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:40.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0a9f9640 1 -- 172.21.15.129:0/2222166738 >> v1:172.21.15.129:6789/0 conn(0x7f0a04144570 legacy=0x7f0a04143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a0a9f9640 1 -- 172.21.15.129:0/2222166738 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a040d2f80 con 0x7f0a041428c0 2024-04-25T00:46:40.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.967+0000 7f0a091f6640 1 -- 172.21.15.129:0/2222166738 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244641167 0 0) 0x7f0a040d2710 con 0x7f0a04144570 2024-04-25T00:46:40.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0a9f9640 1 --2- 172.21.15.129:0/2222166738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a041428c0 0x7f0a04169140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009e50 tx=0x7f09fc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82843166df800c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:40.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a091f6640 1 -- 172.21.15.129:0/2222166738 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0a5830 con 0x7f0a041428c0 2024-04-25T00:46:40.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a091f6640 1 -- 172.21.15.129:0/2222166738 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09fc0a59d0 con 0x7f0a041428c0 2024-04-25T00:46:40.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a091f6640 1 -- 172.21.15.129:0/2222166738 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0a5da0 con 0x7f0a041428c0 2024-04-25T00:46:40.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 -- 172.21.15.129:0/2222166738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a041428c0 msgr2=0x7f0a04169140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/2222166738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a041428c0 0x7f0a04169140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009e50 tx=0x7f09fc01c970 comp rx=0 tx=0).stop 2024-04-25T00:46:40.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 -- 172.21.15.129:0/2222166738 shutdown_connections 2024-04-25T00:46:40.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/2222166738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a041428c0 0x7f0a04169140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 -- 172.21.15.129:0/2222166738 >> 172.21.15.129:0/2222166738 conn(0x7f0a04072be0 msgr2=0x7f0a040d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:40.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 -- 172.21.15.129:0/2222166738 shutdown_connections 2024-04-25T00:46:40.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 -- 172.21.15.129:0/2222166738 wait complete. 2024-04-25T00:46:40.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 Processor -- start 2024-04-25T00:46:40.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.971+0000 7f0a0c45c640 1 -- start start 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 0x7f0a04151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a041428c0 0x7f0a0415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a04144570 0x7f0a0415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a04144050 con 0x7f0a04141010 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a04141600 con 0x7f0a041428c0 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0a041441d0 con 0x7f0a04144570 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 0x7f0a04151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 0x7f0a04151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55576/0 (socket says 172.21.15.129:55576) 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 -- 172.21.15.129:0/1082211624 learned_addr learned my addr 172.21.15.129:0/1082211624 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a099f7640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a041428c0 0x7f0a0415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a9f9640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a04144570 0x7f0a0415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 -- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a04144570 msgr2=0x7f0a0415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a04144570 0x7f0a0415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 -- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a041428c0 msgr2=0x7f0a0415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a041428c0 0x7f0a0415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 -- 172.21.15.129:0/1082211624 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0413fda0 con 0x7f0a04141010 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0a1f8640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 0x7f0a04151eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c500 tx=0x7f0a0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00016020 con 0x7f0a04141010 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a00005220 con 0x7f0a04141010 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a04156e00 con 0x7f0a04141010 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007030 con 0x7f0a04141010 2024-04-25T00:46:40.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a041573c0 con 0x7f0a04141010 2024-04-25T00:46:40.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a04062e50 con 0x7f0a04141010 2024-04-25T00:46:40.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a000071f0 con 0x7f0a04141010 2024-04-25T00:46:40.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f09faffd640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09e80678f0 0x7f09e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:40.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f0a000856f0 con 0x7f0a04141010 2024-04-25T00:46:40.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a099f7640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09e80678f0 0x7f09e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:40.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.975+0000 7f0a099f7640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09e80678f0 0x7f09e8069db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a0415bf30 tx=0x7f09f4007d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:40.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:40.979+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0a00020020 con 0x7f0a04141010 2024-04-25T00:46:41.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.143+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f0a04156a40 con 0x7f0a04141010 2024-04-25T00:46:41.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.151+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f0a00056190 con 0x7f0a04141010 2024-04-25T00:46:41.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.151+0000 7f09faffd640 1 -- 172.21.15.129:0/1082211624 <== mon.0 v2:172.21.15.129:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0a00055a90 con 0x7f0a04141010 2024-04-25T00:46:41.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.151+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09e80678f0 msgr2=0x7f09e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.151+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09e80678f0 0x7f09e8069db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a0415bf30 tx=0x7f09f4007d10 comp rx=0 tx=0).stop 2024-04-25T00:46:41.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 msgr2=0x7f0a04151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 0x7f0a04151eb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c500 tx=0x7f0a0000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 shutdown_connections 2024-04-25T00:46:41.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f09e80678f0 0x7f09e8069db0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a04144570 0x7f0a0415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a041428c0 0x7f0a0415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 --2- 172.21.15.129:0/1082211624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a04141010 0x7f0a04151eb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 >> 172.21.15.129:0/1082211624 conn(0x7f0a04072be0 msgr2=0x7f0a0413d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:41.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 shutdown_connections 2024-04-25T00:46:41.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.155+0000 7f0a0c45c640 1 -- 172.21.15.129:0/1082211624 wait complete. 2024-04-25T00:46:41.174 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:46:41.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:46:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 Processor -- start 2024-04-25T00:46:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 -- start start 2024-04-25T00:46:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 0x7f4d4015c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4d400cd260 0x7f4d400aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4006d1b0 con 0x7f4d400aa0c0 2024-04-25T00:46:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4006d490 con 0x7f4d400cd260 2024-04-25T00:46:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d47b11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4006d770 con 0x7f4d4015bf50 2024-04-25T00:46:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d450ac640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f4d400aa0c0 0x7f4d400ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55630/0 (socket says 172.21.15.129:55630) 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d450ac640 1 -- 172.21.15.129:0/647714140 learned_addr learned my addr 172.21.15.129:0/647714140 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d458ad640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 0x7f4d4015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d460ae640 1 --2- 172.21.15.129:0/647714140 >> v2:172.21.15.196:3300/0 conn(0x7f4d400cd260 0x7f4d400aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389577475 0 0) 0x7f4d4006d1b0 con 0x7f4d400aa0c0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d2c003530 con 0x7f4d400aa0c0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3549017563 0 0) 0x7f4d2c003530 con 0x7f4d400aa0c0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 msgr2=0x7f4d4015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 0x7f4d4015c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 >> v2:172.21.15.196:3300/0 conn(0x7f4d400cd260 msgr2=0x7f4d400aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 --2- 172.21.15.129:0/647714140 >> v2:172.21.15.196:3300/0 conn(0x7f4d400cd260 0x7f4d400aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4006da50 con 0x7f4d400aa0c0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f4d30001680 con 0x7f4d400aa0c0 2024-04-25T00:46:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 >> v1:172.21.15.129:6789/0 conn(0x7f4d400aa0c0 legacy=0x7f4d400ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c0039c0 0x7f4d2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c004810 0x7f4d2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 0x7f4d2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4006d490 con 0x7f4d2c004810 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4006d770 con 0x7f4d2c0039c0 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c00b9d0 con 0x7f4d4015bf50 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d450ac640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c004810 0x7f4d2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d458ad640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c0039c0 0x7f4d2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.295+0000 7f4d460ae640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 0x7f4d2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d450ac640 1 -- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 msgr2=0x7f4d2c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d450ac640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4015bf50 0x7f4d2c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d450ac640 1 -- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c0039c0 msgr2=0x7f4d2c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d450ac640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c0039c0 0x7f4d2c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d450ac640 1 -- 172.21.15.129:0/647714140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c003530 con 0x7f4d2c004810 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d450ac640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c004810 0x7f4d2c004c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c240 tx=0x7f4d3001f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fde8f71f9be257a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30007d40 con 0x7f4d2c004810 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4d30005b00 con 0x7f4d2c004810 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d448ab640 1 -- 172.21.15.129:0/647714140 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30007070 con 0x7f4d2c004810 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d47b11640 1 -- 172.21.15.129:0/647714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c004810 msgr2=0x7f4d2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d47b11640 1 --2- 172.21.15.129:0/647714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c004810 0x7f4d2c004c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c240 tx=0x7f4d3001f410 comp rx=0 tx=0).stop 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d460ae640 1 -- 172.21.15.129:0/647714140 reap_dead start 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d47b11640 1 -- 172.21.15.129:0/647714140 shutdown_connections 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d47b11640 1 -- 172.21.15.129:0/647714140 >> 172.21.15.129:0/647714140 conn(0x7f4d400b14a0 msgr2=0x7f4d400b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d47b11640 1 -- 172.21.15.129:0/647714140 shutdown_connections 2024-04-25T00:46:41.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.299+0000 7f4d47b11640 1 -- 172.21.15.129:0/647714140 wait complete. 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 Processor -- start 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- start start 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c004810 0x7f4d401e1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 0x7f4d401e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d40152740 0x7f4d40152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4006cee0 con 0x7f4d401e7ea0 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4006cd60 con 0x7f4d2c004810 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4015c330 con 0x7f4d40152740 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 0x7f4d401e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 0x7f4d401e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55598/0 (socket says 172.21.15.129:55598) 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 -- 172.21.15.129:0/4056436931 learned_addr learned my addr 172.21.15.129:0/4056436931 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d458ad640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c004810 0x7f4d401e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d460ae640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d40152740 0x7f4d40152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 -- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d40152740 msgr2=0x7f4d40152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d40152740 0x7f4d40152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 -- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c004810 msgr2=0x7f4d401e1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c004810 0x7f4d401e1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 -- 172.21.15.129:0/4056436931 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d401e3ad0 con 0x7f4d401e7ea0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d460ae640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d40152740 0x7f4d40152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d450ac640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 0x7f4d401e8320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d3001f240 tx=0x7f4d2c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30005d70 con 0x7f4d401e7ea0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4d300047f0 con 0x7f4d401e7ea0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d401e3db0 con 0x7f4d401e7ea0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004be0 con 0x7f4d401e7ea0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d401e4650 con 0x7f4d401e7ea0 2024-04-25T00:46:41.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d401e3f80 con 0x7f4d401e7ea0 2024-04-25T00:46:41.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d300ba030 con 0x7f4d401e7ea0 2024-04-25T00:46:41.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d367fc640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4d240678f0 0x7f4d24069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f4d30122870 con 0x7f4d401e7ea0 2024-04-25T00:46:41.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.303+0000 7f4d458ad640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4d240678f0 0x7f4d24069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.307+0000 7f4d458ad640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4d240678f0 0x7f4d24069db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098d0 tx=0x7f4d3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.311+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4d300c1030 con 0x7f4d401e7ea0 2024-04-25T00:46:41.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.499+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d40062e50 con 0x7f4d401e7ea0 2024-04-25T00:46:41.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.499+0000 7f4d367fc640 1 -- 172.21.15.129:0/4056436931 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f4d30019120 con 0x7f4d401e7ea0 2024-04-25T00:46:41.502 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:41.502 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:42:58.001381+0000","modified":"2024-04-25T00:43:01.006515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-25T00:46:41.503 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T00:46:41.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4d240678f0 msgr2=0x7f4d24069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4d240678f0 0x7f4d24069db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098d0 tx=0x7f4d3c008040 comp rx=0 tx=0).stop 2024-04-25T00:46:41.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 msgr2=0x7f4d401e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 0x7f4d401e8320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d3001f240 tx=0x7f4d2c005840 comp rx=0 tx=0).stop 2024-04-25T00:46:41.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 shutdown_connections 2024-04-25T00:46:41.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f4d240678f0 0x7f4d24069db0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d40152740 0x7f4d40152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d401e7ea0 0x7f4d401e8320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 --2- 172.21.15.129:0/4056436931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d2c004810 0x7f4d401e1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 >> 172.21.15.129:0/4056436931 conn(0x7f4d400b14a0 msgr2=0x7f4d400b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 shutdown_connections 2024-04-25T00:46:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.503+0000 7f4d47b11640 1 -- 172.21.15.129:0/4056436931 wait complete. 2024-04-25T00:46:41.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:46:41.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.635+0000 7f4a142e2640 1 Processor -- start 2024-04-25T00:46:41.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.635+0000 7f4a142e2640 1 -- start start 2024-04-25T00:46:41.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a142e2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4a0c0cd260 0x7f4a0c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a142e2640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c06d1b0 con 0x7f4a0c15bf50 2024-04-25T00:46:41.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a142e2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c06d490 con 0x7f4a0c0aa0c0 2024-04-25T00:46:41.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a142e2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c15ca40 con 0x7f4a0c0cd260 2024-04-25T00:46:41.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1207e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f4a0c15bf50 0x7f4a0c15c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55634/0 (socket says 172.21.15.129:55634) 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1207e640 1 -- 172.21.15.129:0/1880051783 learned_addr learned my addr 172.21.15.129:0/1880051783 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1287f640 1 --2- 172.21.15.129:0/1880051783 >> v2:172.21.15.196:3300/0 conn(0x7f4a0c0cd260 0x7f4a0c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405194619 0 0) 0x7f4a0c06d1b0 con 0x7f4a0c15bf50 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f8003530 con 0x7f4a0c15bf50 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1910245108 0 0) 0x7f49f8003530 con 0x7f4a0c15bf50 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 >> v2:172.21.15.196:3300/0 conn(0x7f4a0c0cd260 msgr2=0x7f4a0c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 --2- 172.21.15.129:0/1880051783 >> v2:172.21.15.196:3300/0 conn(0x7f4a0c0cd260 0x7f4a0c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 >> v1:172.21.15.196:6789/0 conn(0x7f4a0c0aa0c0 legacy=0x7f4a0c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c06d770 con 0x7f4a0c15bf50 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f4a08001680 con 0x7f4a0c15bf50 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 >> v1:172.21.15.129:6789/0 conn(0x7f4a0c15bf50 legacy=0x7f4a0c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49f80039c0 0x7f49f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f80051d0 0x7f49f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c15ca40 con 0x7f49f8004810 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49f800c070 con 0x7f49f80039c0 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f49f800c270 con 0x7f49f80051d0 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1187d640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1207e640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49f80039c0 0x7f49f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1287f640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f80051d0 0x7f49f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1187d640 1 -- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f80051d0 msgr2=0x7f49f800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1187d640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f80051d0 0x7f49f800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1187d640 1 -- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49f80039c0 msgr2=0x7f49f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1187d640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49f80039c0 0x7f49f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1287f640 1 -- 172.21.15.129:0/1880051783 reap_dead start 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1187d640 1 -- 172.21.15.129:0/1880051783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f8003530 con 0x7f49f8004810 2024-04-25T00:46:41.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.639+0000 7f4a1207e640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49f80039c0 0x7f49f8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:46:41.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f49fc001780 tx=0x7f49fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92450d579df58e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0013a0 con 0x7f49f8004810 2024-04-25T00:46:41.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f49fc0052f0 con 0x7f49f8004810 2024-04-25T00:46:41.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1107c640 1 -- 172.21.15.129:0/1880051783 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003860 con 0x7f49f8004810 2024-04-25T00:46:41.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/1880051783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8004810 msgr2=0x7f49f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f49fc001780 tx=0x7f49fc01c400 comp rx=0 tx=0).stop 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/1880051783 shutdown_connections 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 --2- 172.21.15.129:0/1880051783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8004810 0x7f49f8004c90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/1880051783 >> 172.21.15.129:0/1880051783 conn(0x7f4a0c0b14a0 msgr2=0x7f4a0c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/1880051783 shutdown_connections 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/1880051783 wait complete. 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 Processor -- start 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- start start 2024-04-25T00:46:41.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f8004810 0x7f4a0c0722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 0x7f4a0c1e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a0c14a870 0x7f4a0c14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c06cd60 con 0x7f4a0c070690 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c06dd30 con 0x7f4a0c14a870 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c06cee0 con 0x7f49f8004810 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 0x7f4a0c1e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 0x7f4a0c1e5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55614/0 (socket says 172.21.15.129:55614) 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 -- 172.21.15.129:0/2495358230 learned_addr learned my addr 172.21.15.129:0/2495358230 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1207e640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f8004810 0x7f4a0c0722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1287f640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a0c14a870 0x7f4a0c14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 -- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f8004810 msgr2=0x7f4a0c0722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f8004810 0x7f4a0c0722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 -- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a0c14a870 msgr2=0x7f4a0c14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a0c14a870 0x7f4a0c14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 -- 172.21.15.129:0/2495358230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c070b10 con 0x7f4a0c070690 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1207e640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f8004810 0x7f4a0c0722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1287f640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a0c14a870 0x7f4a0c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a1187d640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 0x7f4a0c1e5f70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49fc01c230 tx=0x7f49f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0013a0 con 0x7f4a0c070690 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f49fc0ba070 con 0x7f4a0c070690 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c070d90 con 0x7f4a0c070690 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc007a40 con 0x7f4a0c070690 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.643+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c071350 con 0x7f4a0c070690 2024-04-25T00:46:41.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.647+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c062e50 con 0x7f4a0c070690 2024-04-25T00:46:41.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.647+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f49fc007470 con 0x7f4a0c070690 2024-04-25T00:46:41.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.647+0000 7f4a02ffd640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f49f00678c0 0x7f49f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.647+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f49fc120df0 con 0x7f4a0c070690 2024-04-25T00:46:41.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.647+0000 7f4a1207e640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f49f00678c0 0x7f49f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.647+0000 7f4a1207e640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f49f00678c0 0x7f49f0069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a08000fc0 tx=0x7f4a080014d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.651+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4a0c070b10 con 0x7f4a0c070690 2024-04-25T00:46:41.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.831+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a0c075320 con 0x7f4a0c070690 2024-04-25T00:46:41.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.831+0000 7f4a02ffd640 1 -- 172.21.15.129:0/2495358230 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f49fc016110 con 0x7f4a0c070690 2024-04-25T00:46:41.835 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:41.835 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/913638061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":913638061},{"type":"v1","addr":"172.21.15.196:6835","nonce":913638061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/4073213041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":4073213041},{"type":"v1","addr":"172.21.15.129:6835","nonce":4073213041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3597908415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3597908415},{"type":"v1","addr":"172.21.15.129:6837","nonce":3597908415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:42:58.001381+0000","modified":"2024-04-25T00:43:01.006515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/1985283630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1985283630},{"type":"v1","addr":"172.21.15.196:6833","nonce":1985283630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-25T00:46:41.835 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T00:46:41.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f49f00678c0 msgr2=0x7f49f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f49f00678c0 0x7f49f0069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a08000fc0 tx=0x7f4a080014d0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 msgr2=0x7f4a0c1e5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 0x7f4a0c1e5f70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49fc01c230 tx=0x7f49f8005840 comp rx=0 tx=0).stop 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 shutdown_connections 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f49f00678c0 0x7f49f0069d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a0c14a870 0x7f4a0c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c070690 0x7f4a0c1e5f70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 --2- 172.21.15.129:0/2495358230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49f8004810 0x7f4a0c0722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 >> 172.21.15.129:0/2495358230 conn(0x7f4a0c0b14a0 msgr2=0x7f4a0c1596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 shutdown_connections 2024-04-25T00:46:41.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.835+0000 7f4a142e2640 1 -- 172.21.15.129:0/2495358230 wait complete. 2024-04-25T00:46:41.853 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:46:41.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.963+0000 7f6657c1d640 1 Processor -- start 2024-04-25T00:46:41.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.963+0000 7f6657c1d640 1 -- start start 2024-04-25T00:46:41.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f6657c1d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f66500aa0c0 0x7f66500ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f6657c1d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f66500cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f6657c1d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665006d1b0 con 0x7f665015bf50 2024-04-25T00:46:41.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f6657c1d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f665015ca40 con 0x7f66500aa0c0 2024-04-25T00:46:41.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f6657c1d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f665006d490 con 0x7f66500aa550 2024-04-25T00:46:41.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66561ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f66500cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66561ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f66500cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:37234/0 (socket says 172.21.15.129:37234) 2024-04-25T00:46:41.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66561ba640 1 -- 172.21.15.129:0/2229343847 learned_addr learned my addr 172.21.15.129:0/2229343847 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:41.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66551b8640 1 --2- 172.21.15.129:0/2229343847 >> v2:172.21.15.196:3300/0 conn(0x7f66500aa0c0 0x7f66500ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66551b8640 1 -- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 msgr2=0x7f66500cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:41.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66551b8640 1 --2- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f66500cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66551b8640 1 -- 172.21.15.129:0/2229343847 >> v1:172.21.15.196:6789/0 conn(0x7f665015bf50 legacy=0x7f665015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66551b8640 1 -- 172.21.15.129:0/2229343847 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665006d770 con 0x7f66500aa0c0 2024-04-25T00:46:41.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66561ba640 1 --2- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f66500cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:46:41.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.967+0000 7f66551b8640 1 --2- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66500aa0c0 0x7f66500ccb50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f664c0097c0 tx=0x7f664c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e5d7bab9d88d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66549b7640 1 -- 172.21.15.129:0/2229343847 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c0aa5f0 con 0x7f66500aa0c0 2024-04-25T00:46:41.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66549b7640 1 -- 172.21.15.129:0/2229343847 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f664c0af070 con 0x7f66500aa0c0 2024-04-25T00:46:41.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66549b7640 1 -- 172.21.15.129:0/2229343847 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c0aa9d0 con 0x7f66500aa0c0 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66500aa0c0 msgr2=0x7f66500ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66500aa0c0 0x7f66500ccb50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f664c0097c0 tx=0x7f664c01c470 comp rx=0 tx=0).stop 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- 172.21.15.129:0/2229343847 shutdown_connections 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f66500cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2229343847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66500aa0c0 0x7f66500ccb50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- 172.21.15.129:0/2229343847 >> 172.21.15.129:0/2229343847 conn(0x7f66500b14a0 msgr2=0x7f66500b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- 172.21.15.129:0/2229343847 shutdown_connections 2024-04-25T00:46:41.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- 172.21.15.129:0/2229343847 wait complete. 2024-04-25T00:46:41.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 Processor -- start 2024-04-25T00:46:41.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- start start 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 0x7f665014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f6650146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f665015bf50 0x7f6650157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f665006ebe0 con 0x7f66500aa0c0 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f665006ea60 con 0x7f665015bf50 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f665006ed60 con 0x7f66500aa550 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 0x7f665014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 0x7f665014eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55624/0 (socket says 172.21.15.129:55624) 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66551b8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f6650146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 -- 172.21.15.129:0/2758956470 learned_addr learned my addr 172.21.15.129:0/2758956470 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:41.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66561ba640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f665015bf50 0x7f6650157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 -- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 msgr2=0x7f6650146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f6650146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 -- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f665015bf50 msgr2=0x7f6650157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f665015bf50 0x7f6650157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 -- 172.21.15.129:0/2758956470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6650158150 con 0x7f66500aa0c0 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66559b9640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 0x7f665014eeb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f664000c3a0 tx=0x7f664000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66400052d0 con 0x7f66500aa0c0 2024-04-25T00:46:41.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6640005470 con 0x7f66500aa0c0 2024-04-25T00:46:41.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6650156070 con 0x7f66500aa0c0 2024-04-25T00:46:41.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.971+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640007280 con 0x7f66500aa0c0 2024-04-25T00:46:41.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6650156630 con 0x7f66500aa0c0 2024-04-25T00:46:41.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6650062e50 con 0x7f66500aa0c0 2024-04-25T00:46:41.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6640007420 con 0x7f66500aa0c0 2024-04-25T00:46:41.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f66467fc640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f66340678c0 0x7f6634069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:41.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f66551b8640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f66340678c0 0x7f6634069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:41.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f66400851f0 con 0x7f66500aa0c0 2024-04-25T00:46:41.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.975+0000 7f66551b8640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f66340678c0 0x7f6634069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f664c0ae000 tx=0x7f664c0a7000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:41.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:41.979+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6640055580 con 0x7f66500aa0c0 2024-04-25T00:46:42.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.139+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6650075320 con 0x7f66500aa0c0 2024-04-25T00:46:42.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.143+0000 7f66467fc640 1 -- 172.21.15.129:0/2758956470 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16983 (secure 0 0 0) 0x7f6640059880 con 0x7f66500aa0c0 2024-04-25T00:46:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:46:42.146 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":22,"fsid":"aaa4363a-84af-4fd8-b222-fe97e0bf8f21","created":"2024-04-25T00:42:34.024659+0000","modified":"2024-04-25T00:43:39.799123+0000","last_up_change":"2024-04-25T00:42:45.680561+0000","last_in_change":"2024-04-25T00:42:36.346639+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:42:46.680195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:42:56.613344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:42:56.989839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"102816b8-6061-4739-bbba-d256a4352e02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6803","nonce":1911044788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6809","nonce":1911044788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6823","nonce":1911044788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1911044788},{"type":"v1","addr":"172.21.15.129:6817","nonce":1911044788}]},"public_addr":"172.21.15.129:6803/1911044788","cluster_addr":"172.21.15.129:6809/1911044788","heartbeat_back_addr":"172.21.15.129:6823/1911044788","heartbeat_front_addr":"172.21.15.129:6817/1911044788","state":["exists","up"]},{"osd":1,"uuid":"633ce22e-1391-43b9-af52-cae9377ac030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6801","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6802","nonce":1082494394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6805","nonce":1082494394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6811","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6813","nonce":1082494394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6807","nonce":1082494394},{"type":"v1","addr":"172.21.15.129:6808","nonce":1082494394}]},"public_addr":"172.21.15.129:6802/1082494394","cluster_addr":"172.21.15.129:6805/1082494394","heartbeat_back_addr":"172.21.15.129:6813/1082494394","heartbeat_front_addr":"172.21.15.129:6808/1082494394","state":["exists","up"]},{"osd":2,"uuid":"305e1314-691a-443f-b9c1-f78c3fd27f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6825","nonce":4081708389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6827","nonce":4081708389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6831","nonce":4081708389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4081708389},{"type":"v1","addr":"172.21.15.129:6829","nonce":4081708389}]},"public_addr":"172.21.15.129:6825/4081708389","cluster_addr":"172.21.15.129:6827/4081708389","heartbeat_back_addr":"172.21.15.129:6831/4081708389","heartbeat_front_addr":"172.21.15.129:6829/4081708389","state":["exists","up"]},{"osd":3,"uuid":"fdb3a2ff-3539-488c-b764-c8ad0ea448c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6812","nonce":1553174573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6815","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6816","nonce":1553174573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6821","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6822","nonce":1553174573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1553174573},{"type":"v1","addr":"172.21.15.129:6819","nonce":1553174573}]},"public_addr":"172.21.15.129:6812/1553174573","cluster_addr":"172.21.15.129:6816/1553174573","heartbeat_back_addr":"172.21.15.129:6822/1553174573","heartbeat_front_addr":"172.21.15.129:6819/1553174573","state":["exists","up"]},{"osd":4,"uuid":"514c45e0-59e1-432e-bf2e-c2df9ce352cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6801","nonce":3201055999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6803","nonce":3201055999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6811","nonce":3201055999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3201055999},{"type":"v1","addr":"172.21.15.196:6805","nonce":3201055999}]},"public_addr":"172.21.15.196:6801/3201055999","cluster_addr":"172.21.15.196:6803/3201055999","heartbeat_back_addr":"172.21.15.196:6811/3201055999","heartbeat_front_addr":"172.21.15.196:6805/3201055999","state":["exists","up"]},{"osd":5,"uuid":"6b19aaa8-4475-4735-a1d5-f2debb8b5165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6825","nonce":546704147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6827","nonce":546704147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6831","nonce":546704147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":546704147},{"type":"v1","addr":"172.21.15.196:6829","nonce":546704147}]},"public_addr":"172.21.15.196:6825/546704147","cluster_addr":"172.21.15.196:6827/546704147","heartbeat_back_addr":"172.21.15.196:6831/546704147","heartbeat_front_addr":"172.21.15.196:6829/546704147","state":["exists","up"]},{"osd":6,"uuid":"5ce24abd-3fc5-415e-a0e2-08a9a816e4c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6810","nonce":4280234201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6813","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6815","nonce":4280234201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6823","nonce":4280234201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6817","nonce":4280234201},{"type":"v1","addr":"172.21.15.196:6819","nonce":4280234201}]},"public_addr":"172.21.15.196:6810/4280234201","cluster_addr":"172.21.15.196:6815/4280234201","heartbeat_back_addr":"172.21.15.196:6823/4280234201","heartbeat_front_addr":"172.21.15.196:6819/4280234201","state":["exists","up"]},{"osd":7,"uuid":"f7c5504f-d630-40a8-a619-c7c8f7c733f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6809","nonce":221824364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6814","nonce":221824364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6821","nonce":221824364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":221824364},{"type":"v1","addr":"172.21.15.196:6818","nonce":221824364}]},"public_addr":"172.21.15.196:6809/221824364","cluster_addr":"172.21.15.196:6814/221824364","heartbeat_back_addr":"172.21.15.196:6821/221824364","heartbeat_front_addr":"172.21.15.196:6818/221824364","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.938546+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.922349+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.881793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.941095+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.911908+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.902283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.939837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:42:41.917168+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f66340678c0 msgr2=0x7f6634069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f66340678c0 0x7f6634069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f664c0ae000 tx=0x7f664c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 msgr2=0x7f665014eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 0x7f665014eeb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f664000c3a0 tx=0x7f664000c860 comp rx=0 tx=0).stop 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 shutdown_connections 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f66340678c0 0x7f6634069d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66500aa550 0x7f6650146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f665015bf50 0x7f6650157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 --2- 172.21.15.129:0/2758956470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66500aa0c0 0x7f665014eeb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 >> 172.21.15.129:0/2758956470 conn(0x7f66500b14a0 msgr2=0x7f66500cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 shutdown_connections 2024-04-25T00:46:42.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.147+0000 7f6657c1d640 1 -- 172.21.15.129:0/2758956470 wait complete. 2024-04-25T00:46:42.165 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:46:42.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 Processor -- start 2024-04-25T00:46:42.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 -- start start 2024-04-25T00:46:42.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700ccdb0 0x7fb3700723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb37015c7f0 0x7fb37015c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb370072aa0 con 0x7fb3700cd190 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb370070690 con 0x7fb37015c7f0 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb3769e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb3700708e0 con 0x7fb3700ccdb0 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700ccdb0 0x7fb3700723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb37015c7f0 0x7fb37015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700ccdb0 0x7fb3700723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.129:37254/0 (socket says 172.21.15.129:37254) 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36ffff640 1 -- 172.21.15.129:0/185402775 learned_addr learned my addr 172.21.15.129:0/185402775 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36f7fe640 1 -- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700ccdb0 msgr2=0x7fb3700723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36f7fe640 1 --2- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700ccdb0 0x7fb3700723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36f7fe640 1 -- 172.21.15.129:0/185402775 >> v1:172.21.15.196:6789/0 conn(0x7fb3700cd190 legacy=0x7fb370158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.267+0000 7fb36f7fe640 1 -- 172.21.15.129:0/185402775 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb370070bc0 con 0x7fb37015c7f0 2024-04-25T00:46:42.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36f7fe640 1 --2- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb37015c120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3640097c0 tx=0x7fb36401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728b57b9ea2d4781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:42.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36effd640 1 -- 172.21.15.129:0/185402775 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3640b0070 con 0x7fb37015c7f0 2024-04-25T00:46:42.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36effd640 1 -- 172.21.15.129:0/185402775 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb3640ab600 con 0x7fb37015c7f0 2024-04-25T00:46:42.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36effd640 1 -- 172.21.15.129:0/185402775 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3640ab9f0 con 0x7fb37015c7f0 2024-04-25T00:46:42.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 msgr2=0x7fb37015c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 --2- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb37015c120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3640097c0 tx=0x7fb36401c410 comp rx=0 tx=0).stop 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- 172.21.15.129:0/185402775 shutdown_connections 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 --2- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb37015c120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 --2- 172.21.15.129:0/185402775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700ccdb0 0x7fb3700723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- 172.21.15.129:0/185402775 >> 172.21.15.129:0/185402775 conn(0x7fb3700b14a0 msgr2=0x7fb3700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- 172.21.15.129:0/185402775 shutdown_connections 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- 172.21.15.129:0/185402775 wait complete. 2024-04-25T00:46:42.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 Processor -- start 2024-04-25T00:46:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- start start 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 0x7fb3700ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700cd190 0x7fb370141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb370141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb37006ea10 con 0x7fb3700ccdb0 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb37006e890 con 0x7fb37015c7f0 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb3769e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb37006eb90 con 0x7fb3700cd190 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 0x7fb3700ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 0x7fb3700ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55634/0 (socket says 172.21.15.129:55634) 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.271+0000 7fb36ffff640 1 -- 172.21.15.129:0/104575085 learned_addr learned my addr 172.21.15.129:0/104575085 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36f7fe640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700cd190 0x7fb370141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb374f83640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb370141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36ffff640 1 -- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700cd190 msgr2=0x7fb370141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36ffff640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700cd190 0x7fb370141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36ffff640 1 -- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 msgr2=0x7fb370141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36ffff640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb370141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36ffff640 1 -- 172.21.15.129:0/104575085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb370153a90 con 0x7fb3700ccdb0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36ffff640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 0x7fb3700ce4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb36000e7f0 tx=0x7fb36000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360017070 con 0x7fb3700ccdb0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb36000c940 con 0x7fb3700ccdb0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb370153d70 con 0x7fb3700ccdb0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36000cd50 con 0x7fb3700ccdb0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb370154330 con 0x7fb3700ccdb0 2024-04-25T00:46:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb370062e50 con 0x7fb3700ccdb0 2024-04-25T00:46:42.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb36001a070 con 0x7fb3700ccdb0 2024-04-25T00:46:42.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36cff9640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb35c0678c0 0x7fb35c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7fb360085d50 con 0x7fb3700ccdb0 2024-04-25T00:46:42.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36f7fe640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb35c0678c0 0x7fb35c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.275+0000 7fb36f7fe640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb35c0678c0 0x7fb35c069d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ad000 tx=0x7fb364008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:42.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.279+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb3600566b0 con 0x7fb3700ccdb0 2024-04-25T00:46:42.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.455+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb370075320 con 0x7fb3700ccdb0 2024-04-25T00:46:42.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb36cff9640 1 -- 172.21.15.129:0/104575085 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fb360055fb0 con 0x7fb3700ccdb0 2024-04-25T00:46:42.463 INFO:teuthology.orchestra.run.smithi129.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb35c0678c0 msgr2=0x7fb35c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb35c0678c0 0x7fb35c069d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ad000 tx=0x7fb364008040 comp rx=0 tx=0).stop 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 msgr2=0x7fb3700ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 0x7fb3700ce4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb36000e7f0 tx=0x7fb36000ecb0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 shutdown_connections 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fb35c0678c0 0x7fb35c069d80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3700cd190 0x7fb370141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb37015c7f0 0x7fb370141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 --2- 172.21.15.129:0/104575085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3700ccdb0 0x7fb3700ce4b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 >> 172.21.15.129:0/104575085 conn(0x7fb3700b14a0 msgr2=0x7fb37006cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:42.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 shutdown_connections 2024-04-25T00:46:42.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.463+0000 7fb3769e6640 1 -- 172.21.15.129:0/104575085 wait complete. 2024-04-25T00:46:42.483 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:46:42.550 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T00:46:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 Processor -- start 2024-04-25T00:46:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 -- start start 2024-04-25T00:46:42.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ea41437b0 0x7f7ea4162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 0x7f7ea40b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea4143d30 con 0x7f7ea415f2b0 2024-04-25T00:46:42.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea4162b80 con 0x7f7ea41437b0 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7eaa751640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40b6aa0 con 0x7f7ea40b6110 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea3fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f7ea415f2b0 0x7f7ea40b9540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55646/0 (socket says 172.21.15.129:55646) 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea3fff640 1 -- 172.21.15.129:0/441775591 learned_addr learned my addr 172.21.15.129:0/441775591 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea8cee640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 0x7f7ea40b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/441775591 >> v2:172.21.15.196:3300/0 conn(0x7f7ea41437b0 0x7f7ea4162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984084868 0 0) 0x7f7ea4143d30 con 0x7f7ea415f2b0 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e8c003530 con 0x7f7ea415f2b0 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2497882699 0 0) 0x7f7e8c003530 con 0x7f7ea415f2b0 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 msgr2=0x7f7ea40b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 0x7f7ea40b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 >> v2:172.21.15.196:3300/0 conn(0x7f7ea41437b0 msgr2=0x7f7ea4162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 --2- 172.21.15.129:0/441775591 >> v2:172.21.15.196:3300/0 conn(0x7f7ea41437b0 0x7f7ea4162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea4144010 con 0x7f7ea415f2b0 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f7e940016a0 con 0x7f7ea415f2b0 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 >> v1:172.21.15.129:6789/0 conn(0x7f7ea415f2b0 legacy=0x7f7ea40b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 0x7f7e8c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7e8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c004d50 0x7f7e8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4162b80 con 0x7f7e8c003f70 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40b6aa0 con 0x7f7e8c004d50 2024-04-25T00:46:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c00b9d0 con 0x7f7ea40b6110 2024-04-25T00:46:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.595+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7e8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea37fe640 1 -- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 msgr2=0x7f7e8c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea40b6110 0x7f7e8c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea37fe640 1 -- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c004d50 msgr2=0x7f7e8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8c004d50 0x7f7e8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea37fe640 1 -- 172.21.15.129:0/441775591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c003530 con 0x7f7e8c003f70 2024-04-25T00:46:42.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7e8c004810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009ae0 tx=0x7f7e9801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e46c541778a12a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:42.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980b2070 con 0x7f7e8c003f70 2024-04-25T00:46:42.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7e980a56a0 con 0x7f7e8c003f70 2024-04-25T00:46:42.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea2ffd640 1 -- 172.21.15.129:0/441775591 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980096b0 con 0x7f7e8c003f70 2024-04-25T00:46:42.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 -- 172.21.15.129:0/441775591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 msgr2=0x7f7e8c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 --2- 172.21.15.129:0/441775591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7e8c004810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009ae0 tx=0x7f7e9801cab0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7ea8cee640 1 -- 172.21.15.129:0/441775591 reap_dead start 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 -- 172.21.15.129:0/441775591 shutdown_connections 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 -- 172.21.15.129:0/441775591 >> 172.21.15.129:0/441775591 conn(0x7f7ea40b3510 msgr2=0x7f7ea40b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 -- 172.21.15.129:0/441775591 shutdown_connections 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 -- 172.21.15.129:0/441775591 wait complete. 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 Processor -- start 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.599+0000 7f7eaa751640 1 -- start start 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7ea41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea41407c0 0x7f7ea4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea4157d50 0x7f7ea413fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40b9c50 con 0x7f7e8c003f70 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4141470 con 0x7f7ea41407c0 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea415ea30 con 0x7f7ea4157d50 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7ea41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7ea41556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55650/0 (socket says 172.21.15.129:55650) 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 -- 172.21.15.129:0/995997942 learned_addr learned my addr 172.21.15.129:0/995997942 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea41407c0 0x7f7ea4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 -- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea4157d50 msgr2=0x7f7ea413fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea8cee640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea4157d50 0x7f7ea413fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea4157d50 0x7f7ea413fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 -- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea41407c0 msgr2=0x7f7ea4156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea41407c0 0x7f7ea4156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 -- 172.21.15.129:0/995997942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea4155cc0 con 0x7f7e8c003f70 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea41407c0 0x7f7ea4156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea3fff640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7ea41556e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400f390 tx=0x7f7e9400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94007d00 con 0x7f7e8c003f70 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7e9400a0d0 con 0x7f7e8c003f70 2024-04-25T00:46:42.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea4155fa0 con 0x7f7e8c003f70 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400a4e0 con 0x7f7e8c003f70 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea4156560 con 0x7f7e8c003f70 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4062e50 con 0x7f7e8c003f70 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7e94014070 con 0x7f7e8c003f70 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea0ff9640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7e900678c0 0x7f7e90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7e900678c0 0x7f7e90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5969+0+0 (secure 0 0 0) 0x7f7e94088b00 con 0x7f7e8c003f70 2024-04-25T00:46:42.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.603+0000 7f7ea37fe640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7e900678c0 0x7f7e90069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e980af000 tx=0x7f7e98008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:42.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.607+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7e94059400 con 0x7f7e8c003f70 2024-04-25T00:46:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:42.795+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7ea40ba470 con 0x7f7e8c003f70 2024-04-25T00:46:43.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.471+0000 7f7ea0ff9640 1 -- 172.21.15.129:0/995997942 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f7e94058d00 con 0x7f7e8c003f70 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7e900678c0 msgr2=0x7f7e90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7e900678c0 0x7f7e90069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e980af000 tx=0x7f7e98008040 comp rx=0 tx=0).stop 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 msgr2=0x7f7ea41556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7ea41556e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400f390 tx=0x7f7e9400f850 comp rx=0 tx=0).stop 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 shutdown_connections 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f7e900678c0 0x7f7e90069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ea4157d50 0x7f7ea413fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ea41407c0 0x7f7ea4156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 --2- 172.21.15.129:0/995997942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8c003f70 0x7f7ea41556e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 >> 172.21.15.129:0/995997942 conn(0x7f7ea40b3510 msgr2=0x7f7ea40b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:43.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 shutdown_connections 2024-04-25T00:46:43.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.475+0000 7f7eaa751640 1 -- 172.21.15.129:0/995997942 wait complete. 2024-04-25T00:46:43.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:46:43.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa952602640 1 Processor -- start 2024-04-25T00:46:43.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa952602640 1 -- start start 2024-04-25T00:46:43.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa952602640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa94c0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa952602640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c0ced50 con 0x7fa94c06f560 2024-04-25T00:46:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa952602640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c15fe10 con 0x7fa94c1608f0 2024-04-25T00:46:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa952602640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c1453a0 con 0x7fa94c144fc0 2024-04-25T00:46:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa950b9f640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fa94c06f560 0x7fa94c0ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:38438/0 (socket says 172.21.15.129:38438) 2024-04-25T00:46:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa950b9f640 1 -- 172.21.15.129:0/1192216815 learned_addr learned my addr 172.21.15.129:0/1192216815 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:43.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa94c0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692782773 0 0) 0x7fa94c0ced50 con 0x7fa94c06f560 2024-04-25T00:46:43.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa934003530 con 0x7fa94c06f560 2024-04-25T00:46:43.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1229928751 0 0) 0x7fa934003530 con 0x7fa94c06f560 2024-04-25T00:46:43.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 msgr2=0x7fa94c0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa94c0ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 >> v1:172.21.15.196:6789/0 conn(0x7fa94c1608f0 legacy=0x7fa94c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c1600f0 con 0x7fa94c06f560 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7fa944001a00 con 0x7fa94c06f560 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 >> v1:172.21.15.129:6789/0 conn(0x7fa94c06f560 legacy=0x7fa94c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa934003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa934004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934004d50 0x7fa9340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa93400bad0 con 0x7fa9340039f0 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa93400bc80 con 0x7fa934004d50 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa93400be80 con 0x7fa94c144fc0 2024-04-25T00:46:43.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa934003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa950b9f640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934004d50 0x7fa9340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94b7fe640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa934004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 -- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 msgr2=0x7fa934004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa934004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 -- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934004d50 msgr2=0x7fa9340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934004d50 0x7fa9340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94bfff640 1 -- 172.21.15.129:0/1192216815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa934003530 con 0x7fa9340039f0 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa950b9f640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa934004d50 0x7fa9340051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:43.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.623+0000 7fa94b7fe640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c144fc0 0x7fa934004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa94bfff640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa934003e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ac0 tx=0x7fa93c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2cb434b926bb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c009690 con 0x7fa9340039f0 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa93c0a6480 con 0x7fa9340039f0 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa94affd640 1 -- 172.21.15.129:0/1192216815 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a6870 con 0x7fa9340039f0 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 -- 172.21.15.129:0/1192216815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 msgr2=0x7fa934003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 --2- 172.21.15.129:0/1192216815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa934003e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ac0 tx=0x7fa93c01c990 comp rx=0 tx=0).stop 2024-04-25T00:46:43.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa950b9f640 1 -- 172.21.15.129:0/1192216815 reap_dead start 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 -- 172.21.15.129:0/1192216815 shutdown_connections 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 -- 172.21.15.129:0/1192216815 >> 172.21.15.129:0/1192216815 conn(0x7fa94c067760 msgr2=0x7fa94c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 -- 172.21.15.129:0/1192216815 shutdown_connections 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 -- 172.21.15.129:0/1192216815 wait complete. 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 Processor -- start 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.627+0000 7fa952602640 1 -- start start 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa94c13c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa94c15ba00 0x7fa94c0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c0ae1e0 0x7fa94c13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0ce690 con 0x7fa9340039f0 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0ce510 con 0x7fa94c15ba00 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c0ce810 con 0x7fa94c0ae1e0 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa94c13c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa94c13c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39408/0 (socket says 172.21.15.129:39408) 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 -- 172.21.15.129:0/146797909 learned_addr learned my addr 172.21.15.129:0/146797909 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa950b9f640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c0ae1e0 0x7fa94c13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94b7fe640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa94c15ba00 0x7fa94c0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 -- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c0ae1e0 msgr2=0x7fa94c13ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c0ae1e0 0x7fa94c13ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 -- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa94c15ba00 msgr2=0x7fa94c0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa94c15ba00 0x7fa94c0adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 -- 172.21.15.129:0/146797909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c13b780 con 0x7fa9340039f0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94bfff640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa94c13c540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa93c006f90 tx=0x7fa934005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a6d40 con 0x7fa9340039f0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa93c0a7460 con 0x7fa9340039f0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c13ba60 con 0x7fa9340039f0 2024-04-25T00:46:43.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a7870 con 0x7fa9340039f0 2024-04-25T00:46:43.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c13c020 con 0x7fa9340039f0 2024-04-25T00:46:43.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c0ae5f0 con 0x7fa9340039f0 2024-04-25T00:46:43.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa93c0a7a10 con 0x7fa9340039f0 2024-04-25T00:46:43.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa948ff9640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa9380678c0 0x7fa938069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:43.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5969+0+0 (secure 0 0 0) 0x7fa93c120310 con 0x7fa9340039f0 2024-04-25T00:46:43.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94b7fe640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa9380678c0 0x7fa938069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:43.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.631+0000 7fa94b7fe640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa9380678c0 0x7fa938069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa940001600 tx=0x7fa940009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:43.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.635+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa94c0ae5f0 con 0x7fa9340039f0 2024-04-25T00:46:43.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:43.795+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa94c062e50 con 0x7fa9340039f0 2024-04-25T00:46:44.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.475+0000 7fa948ff9640 1 -- 172.21.15.129:0/146797909 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fa93c0ac020 con 0x7fa9340039f0 2024-04-25T00:46:44.476 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:46:44.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa9380678c0 msgr2=0x7fa938069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa9380678c0 0x7fa938069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa940001600 tx=0x7fa940009040 comp rx=0 tx=0).stop 2024-04-25T00:46:44.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 msgr2=0x7fa94c13c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa94c13c540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa93c006f90 tx=0x7fa934005840 comp rx=0 tx=0).stop 2024-04-25T00:46:44.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 shutdown_connections 2024-04-25T00:46:44.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7fa9380678c0 0x7fa938069d80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa94c0ae1e0 0x7fa94c13ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa94c15ba00 0x7fa94c0adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 --2- 172.21.15.129:0/146797909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9340039f0 0x7fa94c13c540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.479+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 >> 172.21.15.129:0/146797909 conn(0x7fa94c067760 msgr2=0x7fa94c142740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:44.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.483+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 shutdown_connections 2024-04-25T00:46:44.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.483+0000 7fa952602640 1 -- 172.21.15.129:0/146797909 wait complete. 2024-04-25T00:46:44.505 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:46:44.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c48cad640 1 Processor -- start 2024-04-25T00:46:44.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c48cad640 1 -- start start 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c48cad640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0c4406f560 0x7f0c44166f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c48cad640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c440ced50 con 0x7f0c441608f0 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c48cad640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4415fe10 con 0x7f0c44144fc0 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c48cad640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0c441453a0 con 0x7f0c4406f560 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0c441608f0 0x7f0c4406ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:38454/0 (socket says 172.21.15.129:38454) 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 -- 172.21.15.129:0/3708793138 learned_addr learned my addr 172.21.15.129:0/3708793138 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c42d9d640 1 --2- 172.21.15.129:0/3708793138 >> v2:172.21.15.196:3300/0 conn(0x7f0c4406f560 0x7f0c44166f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418139200 0 0) 0x7f0c440ced50 con 0x7f0c441608f0 2024-04-25T00:46:44.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c2c003530 con 0x7f0c441608f0 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 58227667 0 0) 0x7f0c2c003530 con 0x7f0c441608f0 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 >> v2:172.21.15.196:3300/0 conn(0x7f0c4406f560 msgr2=0x7f0c44166f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 --2- 172.21.15.129:0/3708793138 >> v2:172.21.15.196:3300/0 conn(0x7f0c4406f560 0x7f0c44166f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 >> v1:172.21.15.196:6789/0 conn(0x7f0c44144fc0 legacy=0x7f0c440ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c441600f0 con 0x7f0c441608f0 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1532463586 0 0) 0x7f0c38001680 con 0x7f0c441608f0 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 >> v1:172.21.15.129:6789/0 conn(0x7f0c441608f0 legacy=0x7f0c4406ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c0039c0 0x7f0c2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c2c004810 0x7f0c2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c2c0051d0 0x7f0c2c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c00c140 con 0x7f0c2c004810 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c00c2f0 con 0x7f0c2c0051d0 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c00c4f0 con 0x7f0c2c0039c0 2024-04-25T00:46:44.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c2c004810 0x7f0c2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 -- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c0039c0 msgr2=0x7f0c2c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4259c640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c0039c0 0x7f0c2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c42d9d640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c2c0051d0 0x7f0c2c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c0039c0 0x7f0c2c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 -- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c2c0051d0 msgr2=0x7f0c2c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c2c0051d0 0x7f0c2c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c41d9b640 1 -- 172.21.15.129:0/3708793138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c003530 con 0x7f0c2c004810 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c42d9d640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c2c0051d0 0x7f0c2c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c42d9d640 1 -- 172.21.15.129:0/3708793138 reap_dead start 2024-04-25T00:46:44.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.627+0000 7f0c4259c640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c0039c0 0x7f0c2c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:46:44.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c2c004810 0x7f0c2c004c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800c1c0 tx=0x7f0c3801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8494653442edf74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38007d00 con 0x7f0c2c004810 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0c380a9020 con 0x7f0c2c004810 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c4159a640 1 -- 172.21.15.129:0/3708793138 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c440ceb10 con 0x7f0c2c004810 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 -- 172.21.15.129:0/3708793138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c2c004810 msgr2=0x7f0c2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c2c004810 0x7f0c2c004c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800c1c0 tx=0x7f0c3801cc20 comp rx=0 tx=0).stop 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 -- 172.21.15.129:0/3708793138 shutdown_connections 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 --2- 172.21.15.129:0/3708793138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c2c004810 0x7f0c2c004c90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 -- 172.21.15.129:0/3708793138 >> 172.21.15.129:0/3708793138 conn(0x7f0c44067760 msgr2=0x7f0c440b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 -- 172.21.15.129:0/3708793138 shutdown_connections 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.631+0000 7f0c48cad640 1 -- 172.21.15.129:0/3708793138 wait complete. 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 Processor -- start 2024-04-25T00:46:44.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- start start 2024-04-25T00:46:44.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c004810 0x7f0c44138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 0x7f0c4415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c4413d470 0x7f0c44139b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44070810 con 0x7f0c44137860 2024-04-25T00:46:44.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44070690 con 0x7f0c4413d470 2024-04-25T00:46:44.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0c44070990 con 0x7f0c2c004810 2024-04-25T00:46:44.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 0x7f0c4415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 0x7f0c4415b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39420/0 (socket says 172.21.15.129:39420) 2024-04-25T00:46:44.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 -- 172.21.15.129:0/1151264754 learned_addr learned my addr 172.21.15.129:0/1151264754 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T00:46:44.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 -- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c004810 msgr2=0x7f0c44138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c004810 0x7f0c44138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 -- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c4413d470 msgr2=0x7f0c44139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c42d9d640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c4413d470 0x7f0c44139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c4413d470 0x7f0c44139b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 -- 172.21.15.129:0/1151264754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c44139840 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c41d9b640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 0x7f0c4415b5b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c380ab040 tx=0x7f0c2c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c380b4070 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0c38000ee0 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4413c990 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38007d00 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4413cf50 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c44062e50 con 0x7f0c44137860 2024-04-25T00:46:44.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c380bf060 con 0x7f0c44137860 2024-04-25T00:46:44.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c32ffd640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0c240678c0 0x7f0c24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:46:44.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c4259c640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0c240678c0 0x7f0c24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:46:44.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.635+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5531+0+0 (secure 0 0 0) 0x7f0c3811eda0 con 0x7f0c44137860 2024-04-25T00:46:44.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.639+0000 7f0c4259c640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0c240678c0 0x7f0c24069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c34007fd0 tx=0x7f0c34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:46:44.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.643+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0c38016970 con 0x7f0c44137860 2024-04-25T00:46:44.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:44.791+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c44075320 con 0x7f0c44137860 2024-04-25T00:46:45.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.483+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1151264754 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f0c380ef900 con 0x7f0c44137860 2024-04-25T00:46:45.484 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' removed 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0c240678c0 msgr2=0x7f0c24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0c240678c0 0x7f0c24069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c34007fd0 tx=0x7f0c34009040 comp rx=0 tx=0).stop 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 msgr2=0x7f0c4415b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 0x7f0c4415b5b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c380ab040 tx=0x7f0c2c005840 comp rx=0 tx=0).stop 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 shutdown_connections 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:6832/2280039152,v1:172.21.15.129:6833/2280039152] conn(0x7f0c240678c0 0x7f0c24069d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c4413d470 0x7f0c44139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c44137860 0x7f0c4415b5b0 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c380ab040 tx=0x7f0c2c005840 comp rx=0 tx=0).stop 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 --2- 172.21.15.129:0/1151264754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c2c004810 0x7f0c44138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:46:45.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.487+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 >> 172.21.15.129:0/1151264754 conn(0x7f0c44067760 msgr2=0x7f0c440afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:46:45.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.491+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 shutdown_connections 2024-04-25T00:46:45.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T00:46:45.491+0000 7f0c48cad640 1 -- 172.21.15.129:0/1151264754 wait complete. 2024-04-25T00:46:45.518 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T00:46:45.518 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:46:45.518 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.528 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:46:45.529 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T00:46:45.529 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.582 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T00:46:45.582 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:46:45.582 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.596 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:46:45.597 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T00:46:45.597 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.652 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T00:46:45.652 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T00:46:45.652 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T00:46:45.652 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.731 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T00:46:45.731 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T00:46:45.731 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.810 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T00:46:45.811 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T00:46:45.811 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:45.884 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T00:46:45.885 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T00:46:45.885 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:51.839 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-25T00:46:51.987 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T00:46:51.987 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T00:46:51.987 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:52.046 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T00:46:52.046 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T00:46:52.046 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:52.144 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T00:46:52.144 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T00:46:52.144 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:52.225 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T00:46:52.225 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T00:46:52.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:57.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-25T00:46:57.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-25T00:46:58.326 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T00:46:58.327 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T00:46:58.327 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T00:46:58.327 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:58.357 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T00:46:58.357 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T00:46:58.357 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:58.386 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T00:46:58.386 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T00:46:58.387 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T00:46:58.387 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:58.412 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T00:46:58.412 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T00:46:58.412 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:58.452 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T00:46:58.452 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T00:46:58.452 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:46:58.507 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T00:46:58.507 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T00:46:58.507 DEBUG:teuthology.orchestra.run.smithi129:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T00:46:58.526 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:46:58.526 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T00:46:58.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:46:58.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:46:58.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:46:58.847 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T00:46:59.132 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:46:59.132 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T00:46:59.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:46:59.364 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T00:46:59.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:46:59.541 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T00:46:59.831 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:46:59.831 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T00:47:00.049 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:47:00.049 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T00:47:00.250 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:47:00.250 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T00:47:00.443 INFO:tasks.ceph:Archiving mon data... 2024-04-25T00:47:00.444 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/data/mon.a.tgz 2024-04-25T00:47:00.445 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-04-25T00:47:00.451 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.KdVYjP2xfL 2024-04-25T00:47:00.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.KdVYjP2xfL 2024-04-25T00:47:00.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.KdVYjP2xfL 2024-04-25T00:47:00.623 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.KdVYjP2xfL is 285KB 2024-04-25T00:47:00.680 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.KdVYjP2xfL 2024-04-25T00:47:00.688 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/data/mon.b.tgz 2024-04-25T00:47:00.688 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-25T00:47:00.694 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.UR2gdijvh1 2024-04-25T00:47:00.694 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.UR2gdijvh1 2024-04-25T00:47:00.834 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.UR2gdijvh1 2024-04-25T00:47:00.903 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.UR2gdijvh1 is 385KB 2024-04-25T00:47:00.964 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.UR2gdijvh1 2024-04-25T00:47:00.969 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/data/mon.c.tgz 2024-04-25T00:47:00.969 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-25T00:47:01.015 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.2t6jrqhoKf 2024-04-25T00:47:01.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2t6jrqhoKf 2024-04-25T00:47:01.146 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.2t6jrqhoKf 2024-04-25T00:47:01.211 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.2t6jrqhoKf is 385KB 2024-04-25T00:47:01.229 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.2t6jrqhoKf 2024-04-25T00:47:01.235 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T00:47:01.235 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:47:01.237 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:47:01.296 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:47:01.339 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:47:01.339 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T00:47:01.340 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/remote/smithi129/crash 2024-04-25T00:47:01.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:47:01.356 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/remote/smithi196/crash 2024-04-25T00:47:01.356 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:47:01.394 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T00:47:01.394 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:47:01.404 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:47:01.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:47:01.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:47:01.534 INFO:tasks.ceph:Compressing logs... 2024-04-25T00:47:01.534 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:47:01.537 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:47:01.562 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-04-25T00:47:01.563 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27717.log 2024-04-25T00:47:01.564 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33983.log 2024-04-25T00:47:01.564 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-04-25T00:47:01.565 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T00:47:01.565 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.27717.log: /var/log/ceph/ceph-client.admin.33983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46050.log 2024-04-25T00:47:01.565 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33983.log.gz 2024-04-25T00:47:01.565 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mgr.y.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27717.log.gz 2024-04-25T00:47:01.565 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46691.log 2024-04-25T00:47:01.567 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5/var/log/ceph/ceph-client.admin.46050.log: --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2024-04-25T00:47:01.567 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27600.log 2024-04-25T00:47:01.567 INFO:teuthology.orchestra.run.smithi129.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46050.log.gz 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46691.log.gz 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45764.log 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33511.log 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.27600.log: /var/log/ceph/ceph-client.admin.45764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33831.log 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27600.log.gz 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45764.log.gz 2024-04-25T00:47:01.568 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46639.log 2024-04-25T00:47:01.570 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33511.log.gz 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33831.log: /var/log/ceph/ceph-client.admin.46639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46874.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33831.log.gz 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46639.log.gz 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33974.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46874.log.gz 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33432.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33987.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33974.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33974.log.gz 2024-04-25T00:47:01.571 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33432.log.gz 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.33987.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34192.log 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45601.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33987.log.gz 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45920.log 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34192.log.gz 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33989.log 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33334.log 2024-04-25T00:47:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45920.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-04-25T00:47:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33989.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45920.log.gz /var/log/ceph/ceph-client.admin.33727.log 2024-04-25T00:47:01.575 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.575 INFO:teuthology.orchestra.run.smithi129.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33989.log.gz 2024-04-25T00:47:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33805.log 2024-04-25T00:47:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45495.log 2024-04-25T00:47:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33727.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33334.log.gz 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33727.log.gz 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33805.log.gz 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35095.log 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46665.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45495.log.gz 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34197.log 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.2.35095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33988.log 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34197.log: /var/log/ceph/ceph-client.admin.33988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34611.log 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34197.log.gz 2024-04-25T00:47:01.578 INFO:teuthology.orchestra.run.smithi129.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33988.log.gz 2024-04-25T00:47:01.579 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45435.log 2024-04-25T00:47:01.579 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.34611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46232.log 2024-04-25T00:47:01.579 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34611.log.gz 2024-04-25T00:47:01.579 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34429.log 2024-04-25T00:47:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46299.log 2024-04-25T00:47:01.581 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45435.log.gz 2024-04-25T00:47:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46232.log: /var/log/ceph/ceph-client.admin.34429.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T00:47:01.581 INFO:teuthology.orchestra.run.smithi129.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34429.log.gz 2024-04-25T00:47:01.581 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46232.log.gz 2024-04-25T00:47:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46299.log.gz 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46821.log 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46743.log: 95.7% -- replaced with /var/log/ceph/ceph-client.2.35095.log.gz 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34455.log 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46535.log 2024-04-25T00:47:01.582 INFO:teuthology.orchestra.run.smithi129.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46821.log.gz 2024-04-25T00:47:01.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34533.log 2024-04-25T00:47:01.585 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34767.log 2024-04-25T00:47:01.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46535.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34455.log.gz 2024-04-25T00:47:01.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46206.log 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46535.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34533.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27835.log 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34185.log 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34767.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46206.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.27835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34507.log 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27835.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34185.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33563.log 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34507.log.gz 2024-04-25T00:47:01.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46509.log 2024-04-25T00:47:01.587 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-04-25T00:47:01.590 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34377.log 2024-04-25T00:47:01.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33563.log: /var/log/ceph/ceph-client.admin.46509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46795.log 2024-04-25T00:47:01.590 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33563.log.gz 2024-04-25T00:47:01.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34377.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33857.log 2024-04-25T00:47:01.590 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46509.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34377.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46795.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46795.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35000.log 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34663.log 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33857.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35000.log: /var/log/ceph/ceph-client.admin.34663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34689.log 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35000.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34663.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.34689.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34689.log.gz 2024-04-25T00:47:01.591 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34194.log 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33589.log 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5/var/log/ceph/ceph-client.admin.34194.log: --verbose -- /var/log/ceph/ceph-client.admin.46769.log 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33589.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34194.log.gz 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33589.log.gz 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45357.log 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3%/var/log/ceph/ceph-client.admin.45357.log: -- replaced with /var/log/ceph/ceph-client.admin.46769.log.gz 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-04-25T00:47:01.594 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45357.log.gz 2024-04-25T00:47:01.595 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T00:47:01.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34822.log: /var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46457.log 2024-04-25T00:47:01.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.1.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz 2024-04-25T00:47:01.597 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-04-25T00:47:01.597 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33909.log 2024-04-25T00:47:01.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-04-25T00:47:01.597 INFO:teuthology.orchestra.run.smithi129.stderr: 87.9%/var/log/ceph/ceph-client.admin.33909.log: -- replaced with /var/log/ceph/ceph-client.admin.46457.log.gz 2024-04-25T00:47:01.597 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33909.log.gz 2024-04-25T00:47:01.598 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34585.log 2024-04-25T00:47:01.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-04-25T00:47:01.598 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz 2024-04-25T00:47:01.599 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34585.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34585.log.gz 2024-04-25T00:47:01.599 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33617.log 2024-04-25T00:47:01.599 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46850.log 2024-04-25T00:47:01.600 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-04-25T00:47:01.600 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33617.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33617.log.gz 2024-04-25T00:47:01.600 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46258.log 2024-04-25T00:47:01.601 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34637.log 2024-04-25T00:47:01.601 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8%/var/log/ceph/ceph-client.admin.46258.log: -- replaced with /var/log/ceph/ceph-client.admin.46850.log.gz 2024-04-25T00:47:01.601 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46258.log.gz 2024-04-25T00:47:01.602 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2024-04-25T00:47:01.602 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34950.log 2024-04-25T00:47:01.603 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34637.log: /var/log/ceph/ceph-client.admin.45383.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34637.log.gz 2024-04-25T00:47:01.603 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46900.log 2024-04-25T00:47:01.604 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.34950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38164.log 2024-04-25T00:47:01.607 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46900.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46900.log.gz 2024-04-25T00:47:01.608 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46076.log 2024-04-25T00:47:01.609 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.38164.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2024-04-25T00:47:01.609 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45706.log 2024-04-25T00:47:01.610 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46076.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46076.log.gz 2024-04-25T00:47:01.611 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-04-25T00:47:01.612 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45706.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45706.log.gz 2024-04-25T00:47:01.612 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34715.log 2024-04-25T00:47:01.613 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46327.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-04-25T00:47:01.614 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:47:01.615 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27731.log 2024-04-25T00:47:01.615 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T00:47:01.615 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27731.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T00:47:01.615 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27851.log 2024-04-25T00:47:01.616 INFO:teuthology.orchestra.run.smithi196.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27731.log.gz 2024-04-25T00:47:01.616 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33660.log 2024-04-25T00:47:01.618 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34715.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34715.log.gz 2024-04-25T00:47:01.618 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33986.log 2024-04-25T00:47:01.618 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.log: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:47:01.618 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46154.log 2024-04-25T00:47:01.620 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27827.log 2024-04-25T00:47:01.620 INFO:teuthology.orchestra.run.smithi196.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27851.log.gz 2024-04-25T00:47:01.620 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.3.33660.log: -- /var/log/ceph/ceph-osd.5.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27899.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.5.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27827.log.gz 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27779.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27875.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27899.log.gz 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27779.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27875.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27779.log.gz 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27875.log.gz 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27755.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33541.log 2024-04-25T00:47:01.621 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27755.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T00:47:01.628 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27755.log.gz 2024-04-25T00:47:01.628 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.33541.log: 95.6% -- replaced with /var/log/ceph/ceph-client.3.33660.log.gz 2024-04-25T00:47:01.628 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:47:01.628 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T00:47:01.628 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.d.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.1.33541.log.gz 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27803.log 2024-04-25T00:47:01.629 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27803.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27803.log.gz 2024-04-25T00:47:01.633 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33986.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33986.log.gz 2024-04-25T00:47:01.633 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33267.log 2024-04-25T00:47:01.633 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46154.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46154.log.gz 2024-04-25T00:47:01.633 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2024-04-25T00:47:01.633 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33267.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33267.log.gz 2024-04-25T00:47:01.633 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46405.log 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45946.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46405.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46405.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34189.log 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45409.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22749.log 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34189.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34189.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45972.log 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.tmp-client.admin.22749.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22749.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45972.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45972.log.gz 2024-04-25T00:47:01.634 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46978.log 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46128.log 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0%/var/log/ceph/ceph-client.admin.46978.log: -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46978.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47030.log 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35145.log 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46128.log.gz 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33935.log 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35145.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47030.log.gz 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45549.log 2024-04-25T00:47:01.635 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35145.log.gz 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46717.log 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45549.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33935.log.gz 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45549.log.gz 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45998.log 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45628.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46717.log.gz 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33701.log 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45998.log: /var/log/ceph/ceph-client.admin.45304.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45998.log.gz 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33458.log 2024-04-25T00:47:01.639 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.640 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34196.log 2024-04-25T00:47:01.640 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33701.log: /var/log/ceph/ceph-client.admin.33458.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33701.log.gz 2024-04-25T00:47:01.640 INFO:teuthology.orchestra.run.smithi129.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33458.log.gz 2024-04-25T00:47:01.640 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45680.log 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34196.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9%/var/log/ceph/ceph-client.admin.45680.log: -- replaced with /var/log/ceph/ceph-client.admin.34196.log.gz 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45680.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47056.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:47:01.642 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T00:47:01.643 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45816.log 2024-04-25T00:47:01.643 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-mds.a.log: 93.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T00:47:01.644 INFO:teuthology.orchestra.run.smithi129.stderr: 93.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46613.log 2024-04-25T00:47:01.644 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T00:47:01.645 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33367.log 2024-04-25T00:47:01.645 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46613.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45816.log.gz 2024-04-25T00:47:01.646 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.46613.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.33985.log 2024-04-25T00:47:01.646 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.647 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33779.log 2024-04-25T00:47:01.647 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33985.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33367.log.gz 2024-04-25T00:47:01.647 INFO:teuthology.orchestra.run.smithi129.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33985.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46180.log 2024-04-25T00:47:01.647 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:01.648 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46353.log 2024-04-25T00:47:01.648 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46180.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33779.log.gz 2024-04-25T00:47:01.648 INFO:teuthology.orchestra.run.smithi129.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46180.log.gz 2024-04-25T00:47:01.648 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33753.log 2024-04-25T00:47:01.650 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46431.log 2024-04-25T00:47:01.650 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46353.log: /var/log/ceph/ceph-client.admin.33753.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46353.log.gz 2024-04-25T00:47:01.650 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33753.log.gz 2024-04-25T00:47:01.650 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33984.log 2024-04-25T00:47:01.650 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46379.log 2024-04-25T00:47:01.650 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46431.log: /var/log/ceph/ceph-client.admin.33984.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46431.log.gz 2024-04-25T00:47:01.652 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45790.log 2024-04-25T00:47:01.652 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33984.log.gz 2024-04-25T00:47:01.652 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46379.log: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33673.log 2024-04-25T00:47:01.652 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46379.log.gz 2024-04-25T00:47:01.653 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34741.log 2024-04-25T00:47:01.653 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33673.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45790.log.gz 2024-04-25T00:47:01.653 INFO:teuthology.orchestra.run.smithi129.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33673.log.gz 2024-04-25T00:47:01.653 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45654.log 2024-04-25T00:47:01.655 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34481.log 2024-04-25T00:47:01.655 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34741.log: /var/log/ceph/ceph-client.admin.45654.log: 88.7% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34741.log.gz 2024-04-25T00:47:01.655 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45654.log.gz 2024-04-25T00:47:01.655 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2024-04-25T00:47:01.655 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46926.log 2024-04-25T00:47:01.656 INFO:teuthology.orchestra.run.smithi129.stderr: 88.0%/var/log/ceph/ceph-client.admin.46587.log: -- replaced with /var/log/ceph/ceph-client.admin.34481.log.gz 2024-04-25T00:47:01.657 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2024-04-25T00:47:01.657 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33647.log 2024-04-25T00:47:01.657 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45461.log 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33647.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46926.log.gz 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33647.log.gz 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33883.log 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35171.log 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33883.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45461.log.gz 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33883.log.gz 2024-04-25T00:47:01.659 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-04-25T00:47:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.35171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46561.log 2024-04-25T00:47:01.661 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35171.log.gz 2024-04-25T00:47:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46952.log 2024-04-25T00:47:01.661 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-04-25T00:47:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34183.log 2024-04-25T00:47:01.662 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46561.log.gz 2024-04-25T00:47:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33537.log 2024-04-25T00:47:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34183.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46952.log.gz 2024-04-25T00:47:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-04-25T00:47:01.663 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34183.log.gz 2024-04-25T00:47:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-04-25T00:47:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45521.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33537.log.gz 2024-04-25T00:47:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34403.log 2024-04-25T00:47:01.665 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-04-25T00:47:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33485.log 2024-04-25T00:47:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34403.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-04-25T00:47:01.666 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46483.log 2024-04-25T00:47:01.666 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34403.log.gz 2024-04-25T00:47:01.666 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34169.log 2024-04-25T00:47:01.667 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46483.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33485.log.gz 2024-04-25T00:47:01.667 INFO:teuthology.orchestra.run.smithi129.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46483.log.gz 2024-04-25T00:47:01.667 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45733.log 2024-04-25T00:47:01.667 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34169.log: /var/log/ceph/ceph-client.admin.45733.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34169.log.gz 2024-04-25T00:47:01.668 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45733.log.gz 2024-04-25T00:47:01.724 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T00:47:01.733 INFO:teuthology.orchestra.run.smithi196.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T00:47:01.842 INFO:teuthology.orchestra.run.smithi129.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T00:47:01.886 INFO:teuthology.orchestra.run.smithi129.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T00:47:01.943 INFO:teuthology.orchestra.run.smithi129.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T00:47:01.953 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T00:47:01.966 INFO:teuthology.orchestra.run.smithi129.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T00:47:01.980 INFO:teuthology.orchestra.run.smithi196.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T00:47:02.138 INFO:teuthology.orchestra.run.smithi196.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T00:47:02.145 INFO:teuthology.orchestra.run.smithi196.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T00:47:02.240 INFO:teuthology.orchestra.run.smithi129.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T00:47:02.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-25T00:47:05.609 INFO:teuthology.orchestra.run.smithi129.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.0.34950.log.gz 2024-04-25T00:47:05.611 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T00:47:05.611 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m4.071s 2024-04-25T00:47:05.611 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m5.991s 2024-04-25T00:47:05.611 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.262s 2024-04-25T00:47:12.007 INFO:teuthology.orchestra.run.smithi196.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T00:47:12.009 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-25T00:47:12.009 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m10.425s 2024-04-25T00:47:12.009 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m11.975s 2024-04-25T00:47:12.009 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.426s 2024-04-25T00:47:12.009 INFO:tasks.ceph:Archiving logs... 2024-04-25T00:47:12.009 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/remote/smithi129/log 2024-04-25T00:47:12.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:47:12.800 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/remote/smithi196/log 2024-04-25T00:47:12.801 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:47:14.329 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:47:14.338 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:47:14.339 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:47:14.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:47:14.417 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:47:14.426 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:47:14.427 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:47:14.429 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:47:14.441 INFO:teuthology.orchestra.run.smithi129.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:47:14.441 INFO:teuthology.orchestra.run.smithi129.stdout:============================================================================== 2024-04-25T00:47:14.442 INFO:teuthology.orchestra.run.smithi129.stdout:*hv01.front.sepi 44.4.53.2 2 u 50 64 377 0.082 -2.090 0.920 2024-04-25T00:47:14.442 INFO:teuthology.orchestra.run.smithi129.stdout:+hv02.front.sepi 63.231.80.2 3 u 44 64 377 0.042 -1.626 1.318 2024-04-25T00:47:14.442 INFO:teuthology.orchestra.run.smithi129.stdout:+hv03.front.sepi 74.6.168.72 3 u 45 64 377 0.088 -0.791 0.951 2024-04-25T00:47:14.442 INFO:teuthology.orchestra.run.smithi129.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:47:14.445 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:47:14.445 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-04-25T00:47:14.445 INFO:teuthology.orchestra.run.smithi196.stdout:*hv01.front.sepi 44.4.53.2 2 u 44 64 377 0.105 -1.760 0.821 2024-04-25T00:47:14.446 INFO:teuthology.orchestra.run.smithi196.stdout:+hv02.front.sepi 63.231.80.2 3 u 41 64 377 0.248 -0.741 0.786 2024-04-25T00:47:14.446 INFO:teuthology.orchestra.run.smithi196.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.126 -0.843 0.780 2024-04-25T00:47:14.446 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:47:14.446 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:47:14.462 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:47:14.464 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:47:14.480 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:47:14.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:47:14.546 INFO:teuthology.task.internal:Duration was 938.276350 seconds 2024-04-25T00:47:14.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:47:14.584 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:47:14.584 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:47:14.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:47:14.624 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:47:14.624 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:47:14.625 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:47:14.671 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T00:47:14.671 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:47:14.681 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:47:14.682 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:47:14.715 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:47:14.751 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:47:14.751 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:47:14.783 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:47:14.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:47:14.862 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:47:14.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:47:14.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:47:14.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:47:14.898 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:47:14.920 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:47:14.929 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-04-25T00:47:14.939 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-25T00:47:14.950 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:47:14.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:47:14.990 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:47:14.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:47:14.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:47:15.007 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:47:15.008 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/remote/smithi129 2024-04-25T00:47:15.008 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:47:15.054 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672276/remote/smithi196 2024-04-25T00:47:15.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:47:15.075 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:47:15.075 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:47:15.096 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:47:15.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:47:15.130 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:47:15.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:47:15.140 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:47:15.140 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:47:15.144 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:47:15.147 INFO:teuthology.orchestra.run.smithi129.stdout: 658467 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:47 /home/ubuntu/cephtest 2024-04-25T00:47:15.166 INFO:teuthology.orchestra.run.smithi196.stdout: 658615 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:47 /home/ubuntu/cephtest 2024-04-25T00:47:15.168 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:47:15.178 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:47:15.200 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/workunit/dir-max-entries} duration: 938.2763500213623 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:47:15.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:47:15.280 INFO:teuthology.run:pass